I have an OKD cluster that is generally running well, but we are struggling with some networking features, especially egress IP and egress firewall behavior. We are considering replacing the current network stack and looked at Cilium Enterprise, but its pricing is much higher than expected. The open-source Cilium option is not suitable for us because of update and maintenance concerns. What affordable CNI plugins or alternative Kubernetes distributions provide comparable features and work reliably with OKD workloads?
4 Answers
OVN-Kubernetes is deeply integrated into OKD, so replacing the CNI is not usually a simple swap. It may be worth considering a different Kubernetes distribution that supports the CNI you want. RKE2 and Talos are examples where Cilium is commonly used, but moving away from OKD also means migrating workloads and rebuilding OKD-specific functionality.
Before changing platforms, check whether the egress IP problems are configuration-related. Common causes include missing egress-assignable node labels, assigning addresses outside the node subnet, or upstream switches not learning the gratuitous ARP when an IP moves. Newer OKD releases have also improved OVN-Kubernetes, so upgrading may help. AdminNetworkPolicy in newer versions can cover some use cases that previously required egress firewall rules.
If you do decide to move to another CNI or distribution, avoid trying to replace the network stack in the existing cluster. Build a separate cluster with the target CNI, test the required egress behavior there, and migrate workloads gradually. That usually carries much less risk than modifying the networking foundation of a running OKD installation.
Antrea is worth investigating as an open-source alternative. It has an OpenShift operator and can provide features such as egress IPs and FQDN-based policies without the same enterprise licensing cost. Calico is another possibility, although features such as egress gateways and DNS policies may require an enterprise subscription, so check the feature matrix carefully.
The migration cost should be included in the comparison. A paid CNI may be expensive, but changing distributions also involves rebuilding integrations, relearning operational workflows, and accepting migration or downtime risk.