Hey everyone,
We're a data center provider and we're currently looking to migrate from our decentralized setup with K3s clusters running at over 20 edge sites. Each of these tiny locations runs services on some lightweight hardware, and we've been able to manage this setup pretty well so far using tools like FluxCD and Grafana for monitoring. However, as we grow—adding more sites every month—it's becoming clear that managing this many clusters isn't sustainable.
I'm curious if switching to EKS Anywhere could help streamline our operations. Specifically, I'm wondering if it would let us reduce the number of clusters we need while still maintaining operational independence and consistent monitoring. Given our increasing scale, I'm also concerned about the load balancing and connectivity challenges between various sites.
Are there any suggestions on consolidating our clusters or insights into using EKS Anywhere effectively? Any alternative solutions that might fit our needs better would also be great!
2 Answers
I looked into EKS Anywhere for your scenario, and it seems the costs could stack up quickly as you scale. It could end up costing you $18k to $24k yearly, which might be a lot given your rapid expansion with 4+ new sites a month. EKS Anywhere isn’t really designed for so many installations—usually, it's for a handful per data center.
Also, about your use case with Grafana Cloud, it's tough on budget for many. Have you considered using something like Ansible to automate the deployment of your clusters instead? That might reduce some of the overhead without all the costs associated with EKS.
Generally, our approach has been to set up each cluster to send metrics directly to Grafana Cloud, which helps us manage monitoring without being overly complicated. Given the size of your sites, it could work out cheaper to do something similar for you.
You know, I think you're on the right track by thinking about automation. For your setup, it might be more beneficial to focus on scripting your server provisioning and using a lightweight orchestration tool. PXE booting followed by a bash or Ansible script is a smart move. That way, you won't really need EKS, and it'll keep costs in check while scaling efficiently.
Just keep refining your GitOps workflows and look for automation tools that fit the small-scale deployments you're handling.
Honestly, I don’t see the need for EKS unless you have the engineering resources for full deployment and automation. It might not fit your current model.