I've been working on a survey summarization tool utilizing Claude Sonnet 4 in AWS Bedrock and decided to test it using AWS Lambda. I noticed something odd: when I run multiple tests within a 2-3 minute interval, the prompt length and input tokens from the previous tests seem to carry over. They all show up in the same log stream in CloudWatch. The only way I've found to reset it is to wait about 5 minutes or redeploy the Lambda function, which then gives me the right token count and prompt length under different log streams. I've gone through my code to reinitialize everything, checked to make sure each Lambda invocation uses a different instance ID, and while it seems like it could be a coding issue on my part, the behavior changes after 5 minutes or with a redeployment. Given the increased cost due to additional tokens, I'm trying to figure out if this is normal behavior or something I should investigate further.
4 Answers
This part doesn't really add up. Can you provide a bit more detail on what exactly is happening?
How are you measuring those tokens? Just double-check that you're not accidentally including extra tokens in your code.
What you're experiencing suggests that the way you're defining your prompts is outside the lambda handler. Variables initialized outside the handler are global to the execution environment and persist until the next cold start. Make sure all your prompts and messages are defined within the handler to reset them on each execution.
Right? This could lead to some bizarre outcomes if you're not careful. Make sure to structure your code right!
It sounds like you're running into the typical behavior of AWS Lambda where it reuses execution environments. Since multiple tests are happening in the same log stream, the environment carries over its state. You can’t really control this directly, so you’ll have to programmatically reset any initialized variables if you want to avoid keeping that state.
Absolutely, OP, take note of this! If you're mixing tokens from previous prompts in future requests, it could confuse users and create unexpected data leaks since there's no sticky session with Lambda.