Hey everyone! I work in a small team of three platform engineers, and we've been using a mix of SaaS monitoring tools like Sentry, New Relic, and Coralogix. Recently, we migrated to AWS and set up EKS with kube-prometheus. Now that we've made this big transition, we're considering our options for monitoring solutions. ClickHouse came up in discussion, and I'm curious to hear from anyone who's installed ClickStack and monitored their Kubernetes clusters with it. How does it compare for developers in terms of ease of use against Grafana, PromQL, or OpenSearch? I'm also wondering about the database management side compared to OpenSearch and PostgreSQL. I'm aware that building something robust can be complex, but I'm keen to understand if replacing Prometheus and OpenSearch would actually offer a better developer experience, manageability, and cost efficiency.
5 Answers
I co-founded a project called KloudMate that runs on ClickHouse and is designed to work intuitively with OpenTelemetry. It supports rapid deployment using the Kubernetes Operator, and it comes with a dedicated module for K8s monitoring right out of the box. You might find it handy if you're looking for something straightforward.
As someone who's dabbled with ClickHouse myself, it's true that it requires a solid understanding of databases, especially when dealing with distributed tables and sharding, plus integrating it with OpenTelemetry can add another layer to the complexity. I appreciate the control it gives, but only recommend it if you have prior experience with distributed systems.
I tried a self-hosted version of SigNoz, which runs on ClickHouse. It wasn't anything spectacular but definitely usable and might suit your needs if you're looking to explore options.
I think managing a self-hosted ClickHouse cluster can be quite challenging and usually requires specific skills. The typical setups with Grafana, Prometheus, or VictoriaMetrics are generally simpler. If you're comfortable with managing Prometheus federations and Elasticsearch indices, you might find some aspects of ClickHouse to be a bit more hands-on, especially regarding disk I/O management and creating your own schemas. It’s good to test with a POC to see if it fits your needs!
For what it's worth, I've found Grafana's user experience to be much more intuitive compared to others. It might make your life a bit easier depending on what you decide.
Are you folks running many cron jobs? I'm struggling to find a solid solution for that kind of workload right now.
Not at my current job, but I had some experience with Argo Workflow. Not sure if that's too much for your needs, but it worked well for me. There are also alternatives like Temporal you could look into.
Yeah, I totally get where you're coming from. I mean, I can manage Prometheus easily with Thanos, but I do worry a bit about ClickHouse's complexities. Good luck with the POC!