I'm working on connecting external tools to an LLM through MCP servers. My first approach was to store API keys in environment variables and let the agent use whichever tool it needed, similar to how I'd configure a conventional backend service.
The concern is that a normal backend has predictable code paths, so it's clear which operation uses a credential. An agent chooses tools dynamically based on its instructions and the current context, meaning the same key could potentially be used for unexpected actions. A secret manager can control storage and rotation, but it doesn't decide whether a particular tool call was authorized.
I've seen recommendations to use separate credentials for each MCP server and inject short-lived credentials at runtime rather than keeping long-lived keys in environment variables. For people operating multi-tool agent systems, what controls are you using? Do you add hard allowlists, proxies, approval steps, or other guardrails, or do you treat the MCP server like an ordinary backend service?
4 Answers
We stopped putting long-lived, full-privilege keys in the agent environment. Our setup uses an intermediary proxy that issues a short-lived OAuth or JWT token for each execution session. The agent never receives the provider’s actual secret.
The proxy enforces tool and parameter allowlists, rate limits, and payload validation before adding the real credential to the outbound request. Read-only tools can run automatically, while mutations, spending, and other high-impact operations require a quick approval. Even if prompt injection causes an unexpected call, the token is narrowly scoped and expires quickly.
I’d treat the MCP layer more like an admission-control or service-mesh boundary than a regular backend. The orchestrator can lease a scoped credential at session start, while the runtime injects it at the edge so the model never sees it. Each outbound request should be logged with the session, tool name, and a hash of the arguments.
Use fine-grained permissions rather than account-wide keys, and enforce the important rules outside the prompt: per-role tool allowlists, approval for destructive calls, idempotency keys for writes, and rejection when a lease is expired or an argument pattern was previously denied. The extra lease step usually adds far less latency than a network call, especially if leases are cached for the session.
How much overhead does the leasing and request stamping add in practice? I’m mainly wondering whether it becomes noticeable for agents making many small calls.
An environment variable is effectively a capability that any code in the process can spend. I’d use a separate identity for every MCP server, grant only the minimum permissions, issue short-lived credentials, and inject them only when the call is made. A vault solves storage and rotation, but it doesn’t authorize the agent’s choice of tool.
For write operations, put a deterministic allowlist in front of the service and use a separate identity from read-only operations. Requiring a person to approve every tool call usually creates too much friction, so reserve approval for destructive or high-impact actions.
That distinction is the important part: secret management tells you where a credential is and when it changes, but it doesn’t determine whether a runtime tool call should be allowed.
A separate policy layer in front of tool dispatch is useful for deciding which calls need review. The rules can combine the tool, the operation type, the target resource, the arguments, the user’s role, and whether the action changes state.
A fixed list of obviously dangerous operations is a good starting point, but it shouldn’t be the only check. Read-only access can usually be automated, while deletion, external messages, purchases, permission changes, and production writes should be blocked or sent for approval. Credentials should still be scoped per server even when a policy layer is present.
That seems more practical than reviewing everything. The approval policy can focus on state changes and sensitive targets instead of slowing down harmless queries.
Short-lived session tokens are especially useful because they limit the damage from accidental exposure or a prompt-injection attempt. Long-lived keys are much harder to contain once they escape their intended path.