I'm currently managing a Kubernetes cluster where we use a combination of certificates from cert-manager and some manually managed TLS Secrets, including older vendor certificates. While cert-manager does a great job with issuing and renewing certs, I'm struggling to gain visibility into a few key areas:
1. How can I identify which certificates are nearing expiration across all namespaces?
2. How do I verify if the renewal process is functioning correctly, since we've experienced some silent failures?
3. How can I track certificates that cert-manager doesn't manage at all?
Right now, I'm using `kubectl get certificates -A` along with jq for parsing, Prometheus with a custom recording rule for monitoring cert-manager certificate expiration times, and performing manual checks on non-cert-manager secrets. While this setup works, it feels unreliable, especially for certs not tracked by cert-manager.
I'm curious about what other people are using for their setups. Specifically:
1. How do you monitor TLS Secrets that aren't classified as Certificate resources?
2. Is anyone using Blackbox Exporter to check endpoints directly? Is the overhead worth it?
3. Do you have alerting systems in place that notify you of renewal failures before they lead to expirations?
We have explored some commercial Certificate Lifecycle Management tools, but they seem excessive for our scale. I'd appreciate hearing about your experiences and solutions!
5 Answers
We use a cert exporter with Prometheus to monitor our certificates. It finds all certs stored as secrets within the cluster and creates metrics for them. This way, you can set up alerts to notify you when cert expiration gets close.
I wrote a tool that scans all DNS names in our Route53 zones and serves them over HTTP Service Discovery to the Prometheus server, which collects the certificate information using Blackbox Exporter. It helps with tracking expiration efficiently.
Though not Kubernetes-specific, I've had great results using Zabbix for monitoring TLS certificates. It fetches the TLS cert for each endpoint, showing when they expire, which has worked really well for all HTTPS workloads, including those on Kubernetes.
I recommend looking into BlackBox Exporter in conjunction with Prometheus. It’s great for probing endpoints directly, especially when you have service discovery enabled in Kubernetes, as it can automatically detect and monitor all ingresses.
Cert-manager provides built-in metrics for monitoring expiration, and we have alerts set up for those metrics. However, we only use cert-manager certificates and don’t track any others.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures