Our company runs a memory-intensive Python application on Kubernetes. Jenkins pulls merged code from GitLab, builds an image, and pushes it to Harbor, while Argo CD deploys the image to the cluster. The application runs with Uvicorn and is exposed through a domain name. Because its memory requirements keep increasing, we are considering moving it to a separate Linux server. I would like to preserve automated deployments from Jenkins and have the application start and run as a managed web service after deployment. What architecture would you recommend for deployment, backups, rapid recovery, and operational safety? Would a dedicated server be better, or should we keep the application in Kubernetes using dedicated resources?
4 Answers
A hybrid approach may be best: move only the memory-heavy service to a dedicated machine while leaving the rest of the system in Kubernetes. Jenkins can build and publish the artifact as before, and Ansible or a controlled deployment script can install the selected version on the server. Keep application data outside the deployment directory, back it up separately, and test restoring both the server configuration and the data. If the service is stateless, rebuilding a host from automation is usually more reliable than restoring the whole machine.
If you do use a standalone server, recreate the safeguards Kubernetes was providing. Run the service under systemd with automatic restarts, health checks, dependency ordering, and a cgroup memory limit such as MemoryMax. Use Ansible or another configuration-management tool to build a repeatable host, and deploy a versioned artifact or golden image rather than modifying the server manually. A reverse proxy such as Nginx or Caddy can handle TLS and forward traffic to Uvicorn.
Before leaving Kubernetes, consider adding a dedicated node with enough RAM. Taint and label the node, then use tolerations and node affinity so only this application runs there. You keep the existing Jenkins and Argo CD pipeline, along with Kubernetes health checks, restart behavior, and easier replacement during maintenance.
That sounds like the most practical option. I had not considered isolating the workload on its own Kubernetes node, but it would preserve the operational features we already use.
The growing memory usage deserves investigation before simply buying a larger machine. Check the number of Uvicorn workers, long-lived connections, sessions, caches, and possible leaks. Profile the application and set realistic memory requests and limits. If the workload is genuinely designed to keep a large in-memory model, document the expected ceiling so you can size the node safely.
The application is AI-related and keeps much of its data in memory, so new features have steadily increased its RAM requirements. I will still investigate whether worker count or a leak is making the growth worse.
A single server loses much of Kubernetes' self-healing. If the host fails, the application is still unavailable, so keep tested recovery procedures and consider a standby host or infrastructure image if the service is critical.