I'm looking for effective ways to aggregate log lines from a Kubernetes container and send them in a file format (or something similar) to external storage. My goal is to avoid sending logs line by line to object storage. Is Fluent-bit a suitable solution for this?
5 Answers
Using Promtail with an S3-backed Loki setup is really solid! If you're looking for a reliable solution, that one definitely delivers.
We’ve had great success with Alloy and Loki too. You can filter the logs based on specific labels that you want to store.
I recently set up a tech stack that worked really well for log aggregation. I used Grafana with Loki and Fluentbit. It was super straightforward and efficient.
Implementing Fluent-bit as a sidecar in your pods has worked well for my team, especially in conjunction with Splunk.
There are plenty of tools available for aggregating logs, like Promtail and Fluentbit. I recommend deploying Fluentbit as a daemonset instead of a sidecar; it's generally a better option unless the sidecar is your only choice. With over 1500 apps running, I’ve only had to use a sidecar for one specific case. Just make sure your application logs are accessible through stdout and stderr, and the daemonset will handle the rest!
I totally agree! Filtering helps keep things tidy and makes it easier to manage logs.