I have about four years of experience as a DevOps engineer—two years at a large company and two at a startup. I've worked with AWS, GCP, Terraform, Kubernetes, observability, CI/CD, and security tools such as Trivy and SonarQube. I've designed and implemented infrastructure, monitoring, infrastructure as code, and security practices from scratch, while also partnering with developers, finding optimization opportunities, and owning projects end to end. I use AI tools such as Claude Code and MCP in my daily work as well.
I'd like to move beyond startup-scale DevOps and learn how infrastructure and platforms are operated at much larger organizations, such as clusters with hundreds of nodes or AI infrastructure. I'm especially interested in GPU scheduling and LLM observability, but I'm unsure whether that's the right direction or what other skills would have the greatest long-term value. What should I focus on to grow in both scale and career progression?
4 Answers
Your next step probably isn’t another tool—it’s learning how systems behave at scale. Focus on large Kubernetes environments, platform engineering, distributed systems, capacity planning, reliability engineering, incident response, and performance tuning. The important shift is understanding why large systems are designed a certain way, not just how to deploy them. Senior engineers create value by making sound tradeoffs, reducing operational risk, and building platforms that many teams can rely on.
A strong path is to build broad foundations first, then go deeper where your work or interests lead. Make sure you’re comfortable with Linux, shell scripting, networking, operating systems, and distributed-systems concepts. It’s also useful to understand how a major CI/CD platform works, including its configuration and scripting model. You don’t need expert-level knowledge of every category immediately; aim to recognize the purpose of each component and how it fits into the overall system.
AI infrastructure is a reasonable area to explore because GPU scheduling, workload orchestration, cost control, and LLM observability are still developing. Just treat it as a specialization built on top of strong fundamentals. Learning how large organizations handle capacity, reliability, security, and operations will remain valuable even if the specific AI tools change.
Try to gain experience with large migrations and modernization projects. Moving an existing team from older infrastructure or processes to a more reliable platform teaches planning, dependency analysis, risk management, training, rollback strategies, and tradeoff evaluation. That kind of work often develops scale-related judgment more effectively than building a greenfield system in a small environment.
That’s the direction I’ve been trying to find. There are plenty of resources showing how to use individual tools, but far fewer explaining how to develop good judgment for operating systems at scale.