We're managing a Kubernetes cluster with over 500 namespaces and 120 nodes, and overall, things have been stable. However, we've recently encountered issues with our open-source NGINX Ingress Controller. During Helm deployments with several dependencies, we're getting admission webhook timeout failures, even after increasing the timeout values. Additionally, after restarts, we keep seeing a 'Sync' scheduled message and delays in configuration loading. When we upgrade NGINX, we often have to delete and recreate all services and ingress resources to avoid getting 'No active endpoints' errors in the logs. Has anyone else experienced similar scaling challenges with the NGINX Ingress Controller? Any tips or advice would be greatly appreciated!
2 Answers
One option you might consider is changing your setup to use a DaemonSet, especially if that works for your workload and depending on how many ingress definitions you're planning to use. Another tip is to group workloads by using ingress classes and pool deployments against each class to prevent any one ingress from being overloaded with configurations. It also sounds like your control plane and ETCD might need scaling, by the way. Just a heads-up— the community version of ingress-nginx is going to be discontinued, so you might want to look into alternatives soon!
Have you tried checking out this resource? It offers some insights into performance optimizations for ingress controllers: sealos.io/blog/sealos-higress-ingress-performance-optimization-minute-to-second. Just a note though—this one focuses on Higress rather than NGINX. But if you're not looking to switch ingress controllers, you might need to find solutions within NGINX for now.
Thanks for the suggestion! But migrating to a new ingress controller isn't something we're considering at this point unless absolutely necessary.