I'm running Kubernetes on AWS with Istio and using Istio's traffic mirroring feature. Previously, a gateway added a unique header before splitting the traffic, so correlating the original and mirrored requests in logs was straightforward. With Istio handling the mirroring, the mirrored request appears to receive a different request ID. What's the recommended production approach for linking the two requests in application or proxy logs?
2 Answers
Istio and Envoy add an x-request-id, but the mirrored request may receive a new value. A common approach is to copy the original request ID into a separate custom header, such as x-origin-request-id, before mirroring. Log that header on both the original and mirrored requests, then use it to correlate them in your log aggregation system. Make sure the applications or sidecars preserve and record the custom header.
You can also use Envoy’s x-request-id as the correlation value if your configuration preserves it across the mirrored request. However, since that header is overrideable and may already have another meaning in your system, a separate header like x-origin-request-id is usually less ambiguous. Whichever option you choose, log both the original request ID and the correlation header consistently.
Keep in mind that x-request-id can be overridden, so it’s safer to use a dedicated correlation header if the existing request ID is already used for another purpose.