I manage enterprise Kubernetes clusters using a monorepo-based Flux bootstrap. Each cluster has one Flux installation, and within that repository I define GitRepository resources under an apps/ directory to pull in other projects. The clusters currently manage several thousand services, and this structure has not caused any issues so far. I'm wondering whether there is a practical limit where a single Flux installation becomes a problem. Should I keep one Flux instance per cluster, or would it be better to run a separate Flux agent for each microservice?
4 Answers
Using the `flux bootstrap` command through Terraform is perfectly valid. The bootstrap command installs and configures Flux in the cluster; the Flux Operator is a different management option, not automatically a performance improvement. It may be useful if you want Kubernetes-native lifecycle management for Flux installations, but switching to it is not necessary just because the cluster manages many services.
Don’t use one Flux installation per microservice. The number of services alone isn’t the important metric; reconciliation workload is. Monitor controller CPU and memory, reconciliation duration, Kubernetes API throttling, and source-fetch failures. If those become bottlenecks, shard the controllers and keep related GitRepository and Kustomization resources on the same shard. With several thousand services and no observed performance problems, there’s no strong reason to change the topology.
Your current split sounds sensible. I wouldn’t add another Flux installation just because the repository or service count grows. Use GitRepository and Kustomization resources to separate applications or teams while keeping the controller set centralized. Consider additional Flux shards or instances when you need a genuine isolation boundary, such as separate credentials, tenant administration, upgrade ownership, or a smaller failure domain.
One Flux installation per cluster is generally the clean default. Multiple GitRepository sources and Kustomizations are designed for this kind of organization, and dependsOn can make dependencies and bootstrap order explicit. Split things only when trust, ownership, or blast-radius requirements justify it.
I’m currently using “bootstrap” to mean the `flux bootstrap` CLI command. Terraform provisions the infrastructure, with Calico and Flux as the main application components installed during cluster creation, and that setup has been working well.