It's been quite a while since I've used Cilium—around 9 months, and my experience with its gateway was filled with networking issues, even though their documentation was solid. On the other hand, it's been over a year since I tried Istio (not in ambient mode), and managing the sidecars was quite the nightmare with an overwhelming number of custom resource definitions. Honestly, I'm not a huge fan of either, but we need a reliable solution for service-to-service communication right now. If you were in my shoes, which one would you prefer? I'm working with a moderately complex microservices setup that includes Kafka in our Kubernetes cluster on EKS, and we've got some AI workloads to consider as well. Time is of the essence!
4 Answers
You might want to give Linkerd a shot. My team found it the easiest to implement compared to Istio and Cilium, leading to fewer complications overall.
Do you actually need the full feature set of a service mesh with complex network policies? If your requirements aren’t that detailed, maybe something simpler like envoy-gateway could work for you. It might fit better without the overhead of Istio or Cilium.
We're not that into complex network policies, but we do need to manage Kafka traffic over UDP and have a solid developer experience for our service-to-service comms. It needs easy service discovery, especially for our local devs who are on K3s right now.
Cilium and Istio have both evolved significantly over the past year. You might find Istio's new Ambient mode useful since it addresses many sidecar limitations. This version uses shared agents on nodes, helping with resource management and overall complexity. If you're looking for visibility and control, it could be worth exploring, although navigating the docs can be tricky.
This is one of the best insights so far. I appreciate it! I might look into that option.
Honestly, if you're ever wondering whether to use Istio, you probably shouldn't. If you really needed it, you wouldn't be asking. It adds a lot of complexity without guaranteed benefits.
I'm surprised to see this is popular. What's the alternative? Handling everything from certs to custom metrics yourself? That sounds like a nightmare. People asking if they should use Istio want to know its benefits, and quite often, I think the answer is yes, especially for production.
This has been my experience too! After trying Istio and Cilium, Linkerd just felt way smoother. I'm just hoping people won't regret this choice down the line.