I recently started a software development company and am now managing a few web applications, specifically one Symfony app, one vanilla PHP app, and one Django app, all hosted on separate VPSs. As our contracts become more complex, I'm realizing the need for robust observability for these applications. I understand that typical practice involves exposing a '/metrics' endpoint for application-level logs, but I'm uncertain about the best ways to secure that endpoint. Additionally, has anyone had success using Tailscale to connect their apps to a Grafana/Prometheus setup? If Tailscale isn't common, what other strategies do people use to ensure secure metric collection across different networks? I'd appreciate any tips as I'm still getting the hang of this!
1 Answer
A common approach is to place your applications behind a reverse proxy like Traefik or HAProxy. Doing so keeps your servers protected, as the observability endpoints won't be directly accessible from the internet. Also, utilizing TLS and authentication methods helps secure your traffic.
I’m a bit confused though. Since my services are on different VPSs (and thus different networks), I’m not sure how the firewall rules would assist in scraping data from Prometheus. If node_exporter is running, it could block public access to port 9901, but how would it let Prometheus scrape metrics if it’s on a different network?