Hi everyone! We're a startup with about 8-10 people in R&D, currently using EKS for our production. For development, we're relying on a Tilt environment that manages a local cluster on each Mac. However, we're running into some issues:
1. The Tilt setup locally is really resource-heavy for a single cluster, making it tough to work efficiently.
2. With the rise of AI coding agents, some of us want to work on branches in parallel using worktrees, but Tilt doesn't support this natively.
We're looking to shift away from this local setup to a cloud-based environment, ideally on AWS. Our DevOps team has started this transition, but the cloud dev environment currently only supports a single tenant. Here are our requirements:
1. It should allow easy provisioning of a cluster or a single namespace that can recycle itself daily.
2. For cost and speed, we'd like to only rebuild modified services, while sharing most other services (like infra and utils) between namespaces, with databases instantiated per namespace.
I've explored options like Metalbear, Signadot, Okteto, DevSpace, and Garden.io, but none seem to fit perfectly. Any recommendations out there? Thanks!
4 Answers
Do you use Kubernetes APIs as part of your product? If not, docker-compose could be a simpler solution for your needs.
Most people address this issue using a shared cluster with separate namespaces for each branch. Take a look at vCluster or DevSpace; they are popular choices for ephemeral dev environments and integrate nicely with EKS.
I had a similar experience with a previous company. Initially, we were using Tilt with a local cluster, but as things scaled, it got cumbersome. I ended up setting up a dedicated VM for each developer with a single-node k3s cluster installed. I configured Tilt to use these VMs, and it worked really well! While working in parallel wasn’t super common, it was definitely doable by simply having multiple source folders and running each against different namespaces.
We created a setup where each developer or branch gets an ephemeral namespace with shared base services, and only the differences get rebuilt. This keeps our costs and spin-up times reasonable even with parallel work. If you're interested, I’d love to share what we do at LeanOps Technologies and see how it compares!
Related Questions
How To: Running Codex CLI on Windows with Azure OpenAI
Set Wordpress Featured Image Using Javascript
How To Fix PHP Random Being The Same
Why no WebP Support with Wordpress
Replace Wordpress Cron With Linux Cron
Customize Yoast Canonical URL Programmatically