How to Manage AWS Credentials for a Python App in Different Environments?

You don't need to change your code! Boto3 will automatically look for credentials in a set order. If you have your keys in a config file on your local machine and assign an IAM role to Batch with the right permissions, Boto3 will use the local keys when running locally and the IAM role's temporary keys in AWS. Just avoid using hardcoded keys in production.

Using hardcoded access keys in production is risky. Boto3 has a built-in credential chain that checks various sources for credentials: environmental variables, the credentials file, and the task role if you're on EC2 or Batch. This way, you don't have to worry about exposing your keys.

To keep things simple, just let Boto3 manage the credentials for you. Use your local credentials file for development, and for Batch jobs, attach an appropriate IAM role. Call `boto3.client('s3')` without any specified credentials, and it will figure out the right ones to use for the environment.

For local testing and CI/CD, consider using LocalStack. It mimics AWS services locally without needing actual credentials. This way, you can build your app without worrying about keys, aligning your local and production setups better.