Hi everyone, I'm seeking practical advice on managing Oracle Kubernetes Engine (OKE). My goal is to implement a zero-downtime Kubernetes upgrade and node replacement while minimizing the risk of issues during node termination. I currently have a node pool with three nodes and I'm considering scaling it up from three to six nodes (fan-out), allowing my workloads to move to the new nodes. After that, I'd cordon and drain the old nodes before scaling back down from six to three nodes (fan-in). My main concern is that in AWS EKS, the documentation states that the oldest instances are terminated first during scale-downs. However, I can't find any information on how OKE handles node terminations during a node pool scale-down. I'm curious if there are documented or observed behaviors for node termination order in OKE and if cordoning or draining old nodes affects which ones are removed during the scale-down process. I'm not treating nodes as pets; I just want to understand the specific behaviors of OKE to minimize risks during controlled upgrades. I'd love to hear from anyone who has experience with this in production OKE clusters. Thanks!
1 Answer
It sounds like you're hitting a common challenge! When you cordon a node, yes, it should evict the pods and they'll be rescheduled onto your new nodes. If you've set up your resources correctly, you shouldn't have issues with outdated services or downtime. Just make sure your deployments are set for your app's availability needs.
What happens if I just drain the old node without deleting it and then scale the node pool down to three from six?