How do you track LLM token spend across services?

For a quick start, log the usage reported in each API response alongside the service name, model, and timestamp. Send it to whatever logging or metrics system you already use, then group it by service and model.

That avoids adding infrastructure, but it only works well if every caller records usage consistently. Check each provider's response format rather than assuming the token fields have the same names.

A gateway is useful when calls are scattered across a bunch of services. Put it in front of the providers and issue a separate virtual key for each service or team. Then you can track usage by key and potentially set limits there too.

The separate keys are the part that makes attribution work. Routing everything through a gateway with one shared key still leaves you with a shared bucket of spend.

Decide what you want to attribute before choosing the tool. A service name might be enough for team budgets, but it won't necessarily tell you which feature or customer is driving usage.

Make attribution metadata part of the request contract, whether you log it in each service or pass it through a gateway. Service, team, environment, and feature are reasonable starting fields. Add customer or account identifiers only if that level of reporting is useful.

Watch out for treating all tokens as if they cost the same. Input and output rates can differ, and caching or batch pricing can make a simple token count times a price table drift from the invoice.

Use the provider's usage and cost reports where available, and reconcile your internal attribution against the provider export each month. Check what level of detail the provider actually exposes before relying on per-key cost reporting.