How can I correlate original and mirrored requests with Istio?
Asked by
MellowCedar42
I'm running Kubernetes on AWS with Istio traffic mirroring and need a reliable way to match each mirrored request with the original request. Previously, a gateway added a unique header before traffic was split, so correlating the two requests in logs was straightforward. Since Istio now handles the mirroring, what approaches work well in production for preserving or propagating a correlation ID?
Answers
A more explicit approach is to copy the incoming request ID into a separate header, such as x-origin-request-id, through the traffic-routing configuration. Log that header in both the original and mirrored services, along with each request’s local ID. This avoids conflicts with x-request-id and makes the relationship easy to search in a log aggregator. Make sure the applications receiving the traffic preserve and log the custom header.
Reply to this answer
Envoy’s x-request-id can help because it may be preserved across the original and mirrored requests. However, it’s configurable and can be overwritten, so don’t rely on it blindly—especially if your system already uses that header for another purpose. Verify the behavior in your Istio and Envoy configuration.
QuietMaple88: Keep in mind that x-request-id is overrideable, so it may not remain stable unless you explicitly configure and validate that behavior.
Reply to this answer
Add your answer
Answers are reviewed before publication.
MellowCedar42: That separate origin ID sounds useful since x-request-id is already used for another purpose in our setup.