Hey everyone! I'm currently working on a project with a pretty straightforward setup including RDS, Redis, and ECS instances, alongside an API Gateway for on-demand image cloning and transfer, and some S3 buckets thrown in.
On my ECS, I've got two constantly running instances (one for application use and another for devs) and a few temporary instances that get triggered by a Java class in my main application container. Most of the tasks I'm handling are async jobs that utilize either 2 or 4GB of memory, mostly aimed at syncing data between our database and external applications, as well as checking for inactive users.
I'm curious, would switching to AWS Lambda instead of using ECS tasks be a better approach? Or is there anything else you would change in my current setup? I sought out some AI advice but really wanted some real-world feedback here!
3 Answers
It really comes down to how intensive your ECS tasks are. If they’re simple and quick, switching to Lambda might be the way to go (just keep in mind the 900s timeout). Also, if you plan to run Lambdas in a custom VPC, you'll need a NAT Gateway or a custom NAT setup for internet access, which could impact your costs depending on your data transfer requirements. I usually lean towards a Lambda-first approach for flexibility, but yes, it does come with some vendor lock-in.
AWS Lambda can be a great fit! I often pair AWS Lambda with DynamoDB for tracking processing statuses. For more intricate workflows, Step Functions help orchestrate various Lambda calls. The combination of Lambda, Step Functions, and S3/DynamoDB has proven to be a powerful and budget-friendly solution for me. In fact, I can process 1TB of data in under 10 minutes using this setup!
Lambdas are fantastic as long as each call is brief. Keeping tasks short is key if you want to maximize performance.
I agree! Just a note though—Lambda is around eight times pricier per CPU cycle compared to EC2. It’s cost-effective because you’re not paying for idle time. If a task takes longer than 10 minutes, you might be better off starting an EC2 instance via Lambda that handles the work and then shuts down afterwards. You could even consider using ECS/Fargate for similar benefits, using the spot market for additional savings!
What are the current best practices for deploying to Lambda?