I'm trying to figure out the best way to work with Kubernetes in local development without losing feature parity with production. My goal is to create an environment that mimics production (specifically EKS) as closely as possible, using tools like k3s and Skaffold. I've set up a custom bash script to manage deployments, but I'm running into issues with authentication and the complexity of managing microservices.
With authentication methods varying between production and local, I'm finding it hard to keep things together. Using Helm makes the situation even messier with its convoluted template syntax. The local development cluster is becoming slower to start as we add more services. Recently, a new Cloud Native PostgreSQL setup broke our local dev environment, and I'm feeling stuck. I'm considering whether it's even realistic to remove feature parity entirely between local and production. I've explored options like Telepresence but am uncertain about their effectiveness. Has anyone successfully tackled this nuance? Should we give up the ambition of having a perfect local environment?
5 Answers
Honestly, I just run everything on an actual cluster. The hassle of local setups isn’t worth it. For debugging, I’ve used both versions of Telepresence, though the v2 is a bit of a pain compared to v1, which no longer works.
How complex is your setup? Local Kubernetes works well with a few services, but as you scale, you have to replicate databases and other dependencies, which can be a pain. Tools like Telepresence can connect your local environment to remote clusters, but you need to think about isolation when multiple developers connect. I'm with Signadot, and we focus on traffic isolation, which might help.
I looked at Signadot but got confused pretty quickly. I’ll dig deeper to see how that could work for us, thanks!
Maintaining parity usually involves instantiating your services and dependencies via containers, but I think trying to replicate every dependency is a lost cause. For external integrations, you might want to explore if they offer sandbox environments instead of spinning everything up locally.
Good point! It’s definitely more complex than just containerization, especially with networking needs and whatnot.
Flux could be useful for you. It allows you to manage multiple environments from a single codebase, including dev and production. For local development, consider using kind with a simple bootstrap script to spin up a local cluster efficiently. Mocking some services might be necessary to keep your local feedback loop short—mimic production without running the whole setup.
I’m curious, how do you manage CRDs and authenticating across environments? We face some issues there as well.
Creating overlays in Kustomize can help with those scenarios.
If you haven't checked it out yet, Skaffold could help with achieving better parity in local dev. I use it with an OrbStack k8s cluster, and it's been a game changer for managing deployments across environments like k3d and minikube.
Yeah, we're already using Skaffold as mentioned in my post, but the challenge is deeper than that.
Is Telepresence stable enough, or are we going to run into a lot of issues if we go for it?