I'm currently using AWS EventBridge Scheduler to trigger 44 schedules per minute, all pointing to a single AWS Lambda function. I typically see 7 to 9 concurrent Lambda invocations at peak times, and all triggers happen within a minute. However, due to some organizational restrictions, I'm not allowed to use Lambda anymore, so I need to migrate this setup to EKS where a containerized microservice will handle the same tasks.
I have a few questions that I hope can help me figure out the right approach:
1. What's the best method to connect EventBridge Scheduler to a microservice running on EKS? Should I expose the service with a LoadBalancer or API Gateway, or is it possible to use a private endpoint for direct invocation?
2. How can I guarantee that all 44 invocations reach my microservice within the same minute as Lambda used to handle? I'm worried about fault tolerance, like pod restarts or scaling issues. Should I set up multiple replicas of the service to manage the traffic better? Are there more reliable or scalable alternatives to EventBridge Scheduler for this?
Any input on architecture patterns, retry handling, or rate limiting to make sure the service performs well under load would be great. I haven't done a proof of concept yet; I'm still determining the best approach.
4 Answers
I wonder what exact payloads are sent to your Lambda through EventBridge. If it’s possible, you may find it simpler to have a pod directly call the microservice instead.
You should definitely look into what events trigger your current Lambda setup. If your schedule consists of configurations triggering multiple APIs, consider using Kubernetes cron jobs instead. This way, you eliminate dependency on external triggers and manage this directly within your pods.
You could have EventBridge trigger an API Gateway that points to your EKS service, or alternatively, use SQS for more robust communication. It’s important to decide how you want to manage the traffic and fault tolerance with your microservice. Using SQS might introduce some asynchronous processing delays, though, so keep that in mind.
Another option could be to route through EventBridge to AWS Batch. This would allow you to run multi-node parallel jobs, which can really help with handling those 44 invocations efficiently.
Related Questions
Set Wordpress Featured Image Using Javascript
How To Fix PHP Random Being The Same
Why no WebP Support with Wordpress
Replace Wordpress Cron With Linux Cron
Customize Yoast Canonical URL Programmatically
[Centos] Delete All Files And Folders That Contain a String