How to Handle Pod Drain with Minimal Downtime in Kubernetes?
Asked by
CoderDude92
I have a workload that typically runs with just one pod. When I need to drain a node, I want to ensure that the existing pod isn't killed right away. Instead, I'd prefer Kubernetes to launch a second pod on another node first, wait for it to be healthy, and only then remove the original pod. This way, downtime will be minimized. Is there a built-in Kubernetes method to achieve this for a single-replica workload, or will I need to create a custom solution?
Answers
This situation can vary based on your workload. If it can handle multiple replicas, opting for a deployment or stateful set with at least two replicas is best. It ensures resilience against failures and unplanned restarts. But if you can't run multiple replicas, you might need a VM with live migration capabilities for better reliability.
Reply to this answer
Before draining the node, try cordoning it, followed by a rollout of your deployment. This way, you can use the maxSurge strategy to temporarily have two replicas running, which will help in maintaining availability during the drain.
Reply to this answer
Set your rolling update parameters to allow a max surge of 1 and max unavailable to 0. First, cordon the node, then perform a rollout restart before finally draining it. This should help ensure minimal disruption.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions