I'm helping a client with a large hybrid environment spanning AWS, Azure, and on-premises systems. They've been using Datadog for several years, but accumulated technical debt has left their APM setup inconsistent. Several libraries are outdated, source instrumentation is misconfigured, and distributed traces no longer connect reliably across important services.
Datadog support confirmed that the underlying fixes need to happen in the application code or local agents. However, coordinating application teams and getting approval to update so many services will take considerable time. We're also developing a centralized observability pipeline to standardize telemetry, but that won't be ready for several months.
In the meantime, is there a practical workaround using a proxy, collector, or transformation layer to intercept telemetry and patch or enrich incomplete spans before they reach Datadog? Has anyone successfully mitigated legacy APM or tracing gaps at the pipeline level without redeploying the applications?
1 Answer
The support guidance is basically correct: a pipeline can transform, tag, sample, or enrich spans that already contain the necessary information, but it generally can’t reconstruct missing parent-child relationships, propagation headers, or instrumentation data that the application never emitted. The durable fix is to update the libraries, instrumentation, and local agents in the affected services.
A collector or proxy could still be useful as a temporary measure for normalization and visibility—for example, adding environment or service metadata, standardizing resource attributes, filtering noisy data, and identifying which services are producing incomplete traces. That can reduce the impact while teams work through the backlog, but it shouldn’t be treated as a way to fully repair broken end-to-end tracing. Since the client already uses coding assistants, a focused automated update effort may be the quickest path if the application teams can allocate review and testing time.
They do have access to coding assistants, but getting the teams to prioritize and review the changes is still the main constraint. A collector-based interim layer may help us standardize what is available while we work through the source fixes.