How to Manage Kubernetes Cluster with Multiple Gateways?
Asked by
TechWizard42
I'm currently learning Kubernetes and have set up a cluster with two networks, each having its own separate gateways. I'm trying to figure out how to configure pods in a specific namespace so that all their external traffic goes through a particular gateway. One of my gateways routes traffic through a VPN via pfsense. I tried using multus to configure a secondary interface for the pods in this namespace, setting default routes for external traffic to use the VPN gateway. However, I'm running into issues with DNS queries—while I want them to go through pfsense, they're still being handled internally, which isn't what I want. It seems like I'm not approaching this correctly. Can anyone share some advice on how to set this up properly?
Answers
Managing multiple gateways in Kubernetes can be tricky. First off, which Container Network Interface (CNI) are you using? If it's Cilium, you could set up an egress gateway. This allows you to designate a specific pod as an exit point for outbound traffic. Just ensure that only this node, which connects to the secondary network, can access it, and you can target only certain namespaces and pods with the egress gateway policy.
Reply to this answer
If you're using Calico as your CNI, then exploring egress gateways is a good way to go. Since you’ve mentioned you've pinned the pods to the host with the secondary interface, you’re on the right track. However, be mindful that pod DNS queries can be tricky, mainly when trying to funnel everything through pfsense while keeping internal cluster DNS functional. It might be worth reviewing your DNS settings or considering a different strategy for your namespace.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions