I’m troubleshooting a canary analysis that appears to abort even though the dashboard looks healthy. What should I check in the Prometheus result handling and Istio metric labels?
Why does my Argo Rollouts canary analysis keep failing?
Answers
Check the value Argo Rollouts receives, not just how the query looks in a graph. A Prometheus query result may be an array of floats rather than a single number, so a condition that compares the whole result directly with a threshold can fail. A condition should account for the returned shape and handle empty or NaN results deliberately before comparing the first value.
Reply to this answer
Also verify the exact Istio label name and value in the metric. In the example discussed, the service label was destination_service, with a fully qualified service name such as node-app-canary.default.svc.cluster.local, rather than a short name under destination_service_name. Confirm what your own Prometheus series actually exposes before changing the template.
Reply to this answer
Don’t rely on the GitOps dashboard alone. A resource can look synced and healthy while an individual analysis run contains the useful failure details. Inspect the rollout and its AnalysisRun status, then compare the query output and condition against the actual result type. That should narrow down whether this is a query, label, or evaluation issue.
Reply to this answer
Add your answer
Answers are reviewed before publication.
Related Discussions