Hi there! I'm managing a Talos-based Kubernetes cluster and I'm exploring the possibility of installing **Istio in Ambient mode** without sidecars. I wanted to reach out to see if anyone here has successfully deployed **Istio Ambient** on a **Talos** cluster. Are there any specific challenges you've encountered due to Talos's immutable and minimal host environment (like the lack of `nsenter` or SSH)? Also, did you have to make any changes to the **CNI setup** (using Flannel, Cilium, or Istio CNI)? Which version of Istio did you find to be most effective, and did you experience any issues getting `ztunnel` or the ambient data plane running smoothly? I've read that Istio 1.15+ has improved support for minimal host OSes, but I haven't found many insights from other Talos users on this yet. I'd really appreciate any experiences, manifests, or tips you have to share! Thanks a lot!
5 Answers
I've been running it in my home lab, initially setting everything up with Istio and Cilium to mirror our work environment. I've removed and reinstalled several times and have been able to run Kiali alongside both Sidecar and Ambient setups. I recently upgraded to 1.27.2, and ensure that the Cilium settings allow for CNI chaining by setting `cni.exclusive=false` in your helm chart. I left kube-proxy in the config; if you remove it, you might hit some failures before Istio is even installed. Here's a sample of my Cilium config for reference!
I tried installing it with Cilium, but everything went sideways after Istio was set up. My whole cluster became unresponsive — not even via Talos or Kubernetes! After multiple attempts with different settings, I just gave up due to the lack of visibility into the issue.
I've set up Istio Ambient with `ztunnel` and it works great, especially if you use version 1.27 or higher since older versions had issues with socket exhaustion. Just make sure to add the right annotations to your namespaces or deployments; it really should just work after that! Oh, and if you're using Cilium as your CNI, stick with Cilium for networking, but let Istio handle the service mesh.
Just a heads up, if you're using Cilium as your CNI, I wouldn’t use it for the mesh itself. Stick with Istio — it provides much better error reporting for maintenance and troubleshooting. Cilium's mesh features are still developing and can lead to confusing connection issues.
Absolutely! I used Talos with Cilium and Istio Ambient as well. Just remember to disable exclusivity on Cilium. Other than that, the setup was pretty smooth! It’s been running on both a production system and my personal dev environment without any major hiccups.
I'd love to hear how it goes for you! If you manage to get it up and running, a PR to the docs would be fantastic — it could help others in the community!
That sounds frustrating! There might be a known bug if you have a pod with `hostNetwork: true`. I faced a similar issue where adding certain namespaces to the mesh made the nodes unreachable due to iptables rules. It could be the reason your setup failed, too.