I'm deciding whether to use Kubernetes for a startup platform and would appreciate lessons from people who have made this decision in production. Assume the team can learn quickly and handle significant complexity—what factors should still influence the choice?
The platform uses single-tenant deployments for multiple customers with different workload sizes. It currently has four microservices. Kubernetes could make deployments, per-customer scaling, and compliance controls easier, but the baseline cost and operational complexity are concerns. We're considering a managed service such as Amazon EKS rather than operating the control plane ourselves.
The more demanding requirements are that users can schedule many jobs across multiple queues, worker processes need to scale as queue volume increases, and we plan to provide isolated code workspaces with GPU support similar to cloud development environments. A fixed number of virtual machines could be either underutilized or insufficient, and implementing scheduling, resource isolation, autoscaling, and GPU placement ourselves may become complicated.
Would Kubernetes be justified here, or would services such as managed queues, autoscaling VMs, ECS/Fargate, serverless workers, Nomad, or another simpler platform be a better starting point?
4 Answers
The four microservices alone would not be a reason to adopt Kubernetes. The scheduled jobs, many queues, per-customer isolation, and GPU workspaces are much stronger reasons because they require scheduling workloads with different resource needs.
That said, separate the question of whether you need orchestration from whether you need Kubernetes specifically. Managed queues plus autoscaling worker groups, ECS/Fargate, or scheduled VM capacity may solve the first version with less operational overhead. Kubernetes becomes more compelling when you have enough workload variety and scale that building those scheduling and isolation features yourself would be a larger risk than operating the platform.
A managed Kubernetes service removes some control-plane work, but it does not make the whole system simple. You still own node pools, upgrades, networking, storage, observability, autoscaling, security policies, GPU drivers, deployment tooling, and incident response. You also need people who can operate it reliably.
If the team has limited Kubernetes experience, build a small non-production prototype first and make sure someone can troubleshoot it before putting an SLA-backed workload on it. Four services on their own could fit comfortably on simpler infrastructure. Don’t spend startup time scaling for a hypothetical customer base before you understand the actual workload.
That’s fair, although the expected enterprise workload is based more on the jobs customers run than on the number of users. A small customer could still submit enough GPU or scheduled work to require autoscaling.
If you choose Kubernetes, use the managed option and keep the first design boring. Use namespaces or another clear boundary for tenants, resource quotas and limit ranges for protection, separate node pools for general and GPU work, and a queue-driven worker model. Use Kubernetes Jobs or CronJobs only where they fit, and let an autoscaler provision capacity instead of maintaining a large idle fleet.
Be careful with the idea of running a separate cluster for every customer. That can become expensive and difficult to upgrade. In many cases, a shared cluster with strong isolation is more manageable, while particularly sensitive customers can receive dedicated infrastructure later.
There are reasonable alternatives. Managed queues can trigger serverless workers or autoscaling VM groups, and ECS/Fargate can run containers without requiring the full Kubernetes ecosystem. For a small number of services, Docker Compose or another lightweight deployment approach may be enough initially.
The tradeoff is that once you need GPU scheduling, workspace isolation, multiple resource classes, burst handling, and many independent job queues, those alternatives may turn into a collection of custom controllers and scripts. Kubernetes is worthwhile when it replaces that growing platform rather than merely because it is the current standard.
That’s the main decision point for us: whether the queue and GPU requirements are already complex enough to justify an orchestrator, or whether we should prove demand with managed cloud primitives first.
That distinction is useful. The concern is that job volume and resource requirements can vary significantly by customer, so I’m trying to avoid creating a custom scheduler that we’ll later have to replace.