How are you routing traffic between on-prem and cloud Kubernetes clusters?

November 19, 2026 4 answers
Asked by 8d!cobblewhisk_472

We have on-prem OpenShift and GKE clusters. Some services run in both environments, while others only run in one. What approaches work for ingress, cross-cluster calls, and failover when a cluster goes down?

Answers

_a59!zordavi.3

I'd separate the external entry point from cross-cluster service traffic. A global load balancer can direct incoming requests to ingress gateways in the appropriate clusters. Health checks and routing policy then determine when traffic moves away from an unhealthy cluster.

For services that exist in only one cluster, make sure the routing rules reflect that. A healthy cluster is not necessarily a valid destination for every service. Avi with its AMKO operator is one paid option for global load balancing.

c.683+ploverlathe:

Failover also depends on what the health check actually tests. A reachable gateway alone does not prove that the service behind it is working.

Reply to this answer
_c74+zeldruno.8

Istio multi-primary is an option if you want services in different clusters to be discoverable through a mesh. That can make cross-cluster calls look more like ordinary service calls, but there is a fair bit to sort out around connectivity, gateways, and policy.

Don't assume an authorization policy configured in one cluster automatically applies everywhere. The details depend on the mesh and trust setup. Separate meshes with federated trust are another possibility, including rules based on workload identity across clusters.

8d!cobblewhisk_472:

Would the mesh also decide which cluster receives incoming traffic, or would that need a separate ingress routing layer?

_d71+vexlora.5:

You still need an entry point and a way to choose where external traffic goes. The mesh setup for service communication doesn't answer that choice by itself.

Reply to this answer
_r67!zunviko.5

Cilium ClusterMesh is worth looking at for cross-cluster communication. Cilium encapsulation can help provide connectivity across clouds, and ClusterMesh offers a way to connect services across clusters.

The catch is that adopting Cilium may mean a networking migration, which could be a substantial project on existing clusters. Check the connectivity and failover behavior against the exact setup rather than assuming a cluster failure will be transparent for every workload.

Reply to this answer
_t57~vexduno.2

Keep cluster management and traffic routing as separate questions. A tool that provisions or manages clusters is not necessarily the thing that decides which cluster receives a request or connects services between clusters.

That distinction helps narrow down whether you need global load balancing, cross-cluster networking, a service mesh, or some combination.

Reply to this answer

Add your answer

Answers are reviewed before publication.

Related Discussions