I'm planning to deploy Karpenter on an Amazon EKS cluster and would appreciate feedback on the architecture. We currently use an app-of-apps setup with Argo CD, and everything runs on a managed node group (MNG).
The proposed design is to reduce the managed node group to a small, static pool and run the Karpenter controller there alongside Argo CD and other essential cluster components. Karpenter would then provision and manage capacity for application workloads. Karpenter itself would be installed and managed through Argo CD as part of the cluster bootstrap process.
Is this a sound design, or are there important changes or best practices we should consider? In particular, I'm interested in avoiding bootstrap dependencies, ensuring Karpenter remains available during node changes, and deciding which system components should stay on the static node group.
3 Answers
One clarification: Karpenter does not need Argo CD in order to function. Argo CD is just the tool managing the Kubernetes manifests or Helm release. Treat Karpenter as an independent controller with its own bootstrap requirements, and make sure the installation process can bring it up on the static capacity before expecting it to provision the rest of the cluster.
A practical setup is to taint the managed node group so ordinary workloads do not drift onto it. Add tolerations only for Karpenter, Argo CD, and the specific bootstrap or control-plane support components that belong there. Otherwise the supposedly temporary static group can become a permanent dependency because application workloads keep consuming its capacity.
Use PodDisruptionBudgets for important Argo CD components, particularly the repository service. Without disruption protection, consolidation or node maintenance can terminate a pod in the middle of a synchronization and make recovery more painful.
That architecture is generally sound. Keep a small, stable managed node group for Karpenter, Argo CD, and only the cluster components needed for recovery and bootstrapping. Karpenter should not depend on nodes that it provisions or manages, otherwise a scale-down or replacement event can remove the controller it needs to recover.
Managing the Karpenter installation with Argo CD is fine, but Argo CD and Karpenter need stable capacity before dynamic workloads can be created. Avoid putting every system workload on the static group; reserve it for genuinely critical services and prevent normal application workloads from landing there.
The main design work is in the NodePool and EC2NodeClass configuration: use enough instance flexibility, configure availability zones, decide how Spot and On-Demand capacity should be mixed, set sensible disruption and consolidation policies, apply resource limits, and ensure workloads have accurate resource requests plus appropriate affinity and topology rules.
Also, avoid relying on a single bootstrap node. A couple of small static nodes spread across availability zones, with suitable disruption protection, gives Karpenter and Argo CD a much better chance of remaining available during maintenance or node replacement.
It may also be worth considering Fargate for a few truly cluster-critical pods if that fits your operational model. The goal is the same: make sure Karpenter and the deployment controller are not disrupted by the node-management system they operate.