How can I track TLS certificate expiration in my Kubernetes cluster?
Asked by
CuriousCoder42
I'm currently managing a Kubernetes cluster where we use a combination of certificates from cert-manager and some manually managed TLS Secrets, including older vendor certificates. While cert-manager does a great job with issuing and renewing certs, I'm struggling to gain visibility into a few key areas:
1. How can I identify which certificates are nearing expiration across all namespaces?
2. How do I verify if the renewal process is functioning correctly, since we've experienced some silent failures?
3. How can I track certificates that cert-manager doesn't manage at all?
Right now, I'm using `kubectl get certificates -A` along with jq for parsing, Prometheus with a custom recording rule for monitoring cert-manager certificate expiration times, and performing manual checks on non-cert-manager secrets. While this setup works, it feels unreliable, especially for certs not tracked by cert-manager.
I'm curious about what other people are using for their setups. Specifically:
1. How do you monitor TLS Secrets that aren't classified as Certificate resources?
2. Is anyone using Blackbox Exporter to check endpoints directly? Is the overhead worth it?
3. Do you have alerting systems in place that notify you of renewal failures before they lead to expirations?
We have explored some commercial Certificate Lifecycle Management tools, but they seem excessive for our scale. I'd appreciate hearing about your experiences and solutions!
Answers
Cert-manager provides built-in metrics for monitoring expiration, and we have alerts set up for those metrics. However, we only use cert-manager certificates and don’t track any others.
Reply to this answer
I recommend looking into BlackBox Exporter in conjunction with Prometheus. It’s great for probing endpoints directly, especially when you have service discovery enabled in Kubernetes, as it can automatically detect and monitor all ingresses.
Reply to this answer
Though not Kubernetes-specific, I've had great results using Zabbix for monitoring TLS certificates. It fetches the TLS cert for each endpoint, showing when they expire, which has worked really well for all HTTPS workloads, including those on Kubernetes.
Reply to this answer
I wrote a tool that scans all DNS names in our Route53 zones and serves them over HTTP Service Discovery to the Prometheus server, which collects the certificate information using Blackbox Exporter. It helps with tracking expiration efficiently.
Reply to this answer
We use a cert exporter with Prometheus to monitor our certificates. It finds all certs stored as secrets within the cluster and creates metrics for them. This way, you can set up alerts to notify you when cert expiration gets close.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions