I'm on a three-person team building a prototype AI application: one data scientist and two engineers. Between the engineers, we have roughly a year of combined Python experience. I've also had to learn Docker, GitLab CI, Git, and general software engineering practices while building the project.
Our previous setup used Docker Compose. We had a development configuration with a hot-mounted source directory for quick changes, while staging and production used images built by CI and pushed to our registries. Environment-specific settings were kept in ignored files with templates, and Makefile commands simplified the workflow.
We recently lost access to our on-premises server and now have to deploy to a k3s cluster managed by another team. That team is also still learning Kubernetes. I've managed to get our service deploying with Helm and Argo CD, and created an infrastructure repository containing environment-specific charts, values, and manifests translated from our Compose setup.
Staging and production seem manageable: CI builds images, updates image tags and namespaces in the infrastructure repository, and Argo CD deploys them. The part I'm unsure about is development. We normally have broad access to the source code and infrastructure, but we don't yet have our own cluster or namespace access. How should each engineer run an individual development instance, view logs, and make quick code changes without disrupting staging or production? Should we use separate namespaces and image tags, local Kubernetes clusters, a synchronization tool, or some other workflow? Any practical guidance for making this manageable for a small, relatively inexperienced team would be appreciated.
3 Answers
A practical Kubernetes development setup would give each engineer an isolated namespace, or at least an isolated release name within a shared development namespace. Each person can deploy their own Helm values with a unique image tag, resource limits, and hostname. CI can build those branch images, while Helm or Argo CD updates only that developer’s release.
For daily work, use commands such as port-forwarding to reach a service locally and the normal pod-log and shell commands to inspect it. If rebuilding and pushing an image for every small change is too slow, tools such as Skaffold, Tilt, or DevSpace can watch files, rebuild images, synchronize changes, and redeploy automatically. Keep staging and production controlled by CI and protected from direct developer changes.
While access is being sorted out, run a local cluster with kind or another lightweight Kubernetes distribution. It’s useful for testing charts, values files, services, probes, config maps, and ingress without waiting on the shared cluster. Make the local workflow use the same Helm charts as the real environments, with only a small local values override.
For application development, you may not need every dependency inside Kubernetes. You can run the code locally and connect it to selected development services in the cluster through port-forwarding, or run only the components that need to be tested in Kubernetes. This usually gives faster feedback than rebuilding the entire application for every edit.
I’ve started looking into kind and am working through Kubernetes networking concepts. A local cluster seems like a good way to learn and validate the deployment setup while the infrastructure team finishes configuring access.
Before investing too heavily in Kubernetes, make sure it’s actually required for this prototype. A managed application platform or a simpler container service might be easier for a small team with limited Kubernetes experience. If the organization has already committed to k3s, though, ask the platform team for a dedicated development namespace with appropriate permissions rather than trying to share staging or production.
That makes sense. The cluster decision is mostly out of our hands, so our immediate goal is to establish a safe and repeatable workflow within that environment.
Separate namespaces would also make permissions and cleanup much clearer. I’d ask the cluster team for a reusable namespace template, quotas, network policies, and a documented way to request or delete temporary environments.