I'm planning a production cluster with three control plane nodes and nine workers. Is there a good reason to put a hardware appliance in front of the API, or would kube-vip or HAProxy VMs be enough?
What would you use to load balance the Kubernetes API on-prem?
Answers
I'd start with kube-vip if you want a stable control plane address without adding more VMs to manage. Running it on the control plane nodes keeps the setup fairly compact, and ARP mode is one option if it fits your network.
Check the failover behavior in your environment, especially what happens to the VIP when a control plane node goes away. A reachable VIP doesn't by itself guarantee that the API server or etcd is healthy.
Reply to this answer
HAProxy on separate VMs is a reasonable choice if you prefer a load balancer you can inspect without relying on the cluster nodes. That separation can make troubleshooting cleaner.
The tradeoff is more infrastructure to maintain. Make sure the HAProxy endpoint itself has a failover plan, rather than putting a single VM in front of an otherwise redundant control plane.
Reply to this answer
First decide whether you need a reliable API address, distribution across API servers, or both. A VIP failover setup and a dedicated load balancer can solve different parts of that problem, depending on how they're configured.
BGP-based load balancing is another option if your network supports it, though that brings its own network configuration and operational dependencies.
Reply to this answer
A hardware appliance isn't automatically wrong, but for a cluster this size it may be more machinery and process than you need. I'd consider it if your organization already operates one and its capabilities or support model solve a specific requirement. Otherwise, kube-vip or HAProxy are sensible places to look first.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Tavlix-7q:
Also try the two-node loss case. With three etcd members, losing two means losing quorum, even if the remaining node can still hold the VIP.