How can each application use a dedicated static IP for ingress and egress?
Asked by
MellowQuasar42
We run a small five-worker-node RKE2 cluster with Calico, Istio, and MetalLB. Istio handles mTLS and ingress, while MetalLB provides the load balancer IP pool. Our networking team wants each of roughly 30 applications to have one static IP shared by both inbound and outbound traffic, so they can create tightly scoped firewall rules for external services. What is the best way to make each application's outbound traffic use a specific source IP? Most of the solutions I have found rely on dedicated egress nodes, but with only five workers, I am not sure how practical that is for dozens of distinct egress IPs. Is there a better approach, such as Calico egress gateways, NAT, or an external network appliance?
Answers
Calico's egress gateway features are a reasonable place to start. You can assign workloads to specific egress gateways and have their traffic SNAT to addresses configured for those gateways. However, check the capabilities of your Calico edition carefully: the open-source version may not provide the same failover and high-availability behavior as an enterprise setup. You may need dedicated nodes, labels, taints, and tolerations to keep workloads from landing on the gateway nodes themselves, and the chosen egress addresses must be configured on the node interfaces, either manually or through automation.
Reply to this answer
An external NAT device or network appliance could be simpler than managing dozens of gateway addresses inside a small cluster. Route selected application traffic through that appliance and configure source NAT there. This can centralize firewall policy and may offer better redundancy than implementing one egress path per application in Kubernetes.
Reply to this answer
The most maintainable answer may be to push back on the one-IP-per-application requirement and request a small CIDR range instead. Firewalls generally support rules for CIDRs or IP ranges, and using a single address for each path can create unnecessary single points of failure. It would also be wise to use separate addresses for ingress and egress rather than coupling them.
Reply to this answer
Other Kubernetes networking solutions can also provide egress gateway behavior. For example, an egress policy can associate namespaces or workloads with gateway nodes and fixed source addresses. Keep in mind that each distinct address usually requires an address configured on a node or virtual interface, and concentrating traffic on specific nodes reduces topology-based redundancy. With only five workers, a design using a CIDR range or a smaller number of shared egress addresses will likely be easier to operate and keep highly available.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions