I launch Argo Workflows through Hera, and the workflow runs successfully. I can inspect its outputs, but I have not found a straightforward way to retrieve the task logs in my Python script. For now, I can read them with kubectl. Is that the practical option, or is there a better setup for showing useful output to less technical users?
What’s the best way to show Argo workflow logs from a Hera script?
Asked by
e2+Switchboard
Answers
Think of Argo as running the workflow, not as your long-term log store. Have tasks write useful messages to standard output, then collect container logs with your cluster’s logging system, such as Fluent Bit forwarding to Loki or another backend. Your script can then fetch logs from that system if you want to present them directly.
Argo log archiving may help with access through Argo, but it is not a full replacement for a dedicated logging pipeline.
Reply to this answer
If you only need logs right after a run, calling the Kubernetes API to read the pod logs is a reasonable approach. It does not have to mean shelling out to kubectl. Check whether the Kubernetes Python client fits your setup, and make sure the identity used by the script is allowed to read pod logs in the relevant namespace.
Reply to this answer
There are two separate questions here: where the logs are stored, and whether the caller has permission to read them. If the output is already on standard output and kubectl can display it, check the service account or user credentials your Python code uses for permission to get pod logs. Argo’s workflow log view can also be affected by those permissions.
Reply to this answer
For an end-user tool, I would avoid making raw logs the only feedback. Return a short success or failure summary from the workflow, and include a link or reference to the full logs in your logging system. That keeps the immediate response understandable without treating transient pod logs as the permanent record.
Reply to this answer
Add your answer
Answers are reviewed before publication.
e2+Switchboard:
That makes sense. I was hoping Hera itself would return the output, but reading pod logs through the API may be the simplest route for immediate feedback.