Hey everyone, I wanted to share a method our team has been perfecting and see what you all think, especially since testing microservices on Kubernetes can be tricky. Traditionally, we faced two choices: rely on a constantly broken shared staging environment or spin up a complete environment for every pull request, which can be slow and expensive as projects grow. We're trying a new approach using a service mesh like Istio or Linkerd to create lightweight, request-level ephemeral environments within a shared cluster. Basically, we only deploy the modified services in the shared dev/staging cluster. When we run tests, we inject a unique HTTP header into the request that the service mesh uses to route traffic to the new service version while preserving the stable baseline for unaffected services. This allows us to have an isolated test context without duplicating the entire stack. I'm also a co-founder at Signadot, where we've incorporated this into our product. Would love to exchange insights if you're working on something similar with Istio, Linkerd, or advanced Ingress rules! Thanks!
2 Answers
Is there any specific setup needed for the applications themselves? Like, do they have to handle header inspection or something similar?
How does this approach work for services that aren't using HTTP? For instance, if some services communicate via Kafka or other systems where you can't control downstream routing in the same way?
For asynchronous flows, instead of routing requests, you can selectively consume messages. The publisher can propagate the headers into the messages they publish. Then the consumers need to check those headers to match with their service version. They can get the mapping of headers to service versions from a central service. I wrote more about this concept on our blog.
Yes, the services need to propagate those headers from input requests to output calls. This is usually done with OpenTelemetry libraries to ensure everything works correctly.