I've been diving into Step Function Express lately, and for a lot of my workflows across different microservices, they seem to replace lambdas entirely while offering benefits like reduced costs, no cold starts, and no concurrency limits. However, I've run into a major limitation: they can't run inside a VPC. This prevents me from using a Valkey / Redis cluster and accessing other internal services in private subnets. Are there any plans to enable Step Function Express to operate within a VPC like you can with Lambdas? I'm also interested in hearing how you all utilize Step Functions in your microservices and workflows.
2 Answers
You can start Step Functions with lambdas, so you might be able to manage your VPC flows through those. Personally, many of my Step Function workflows just call lambdas directly.
Welcome to the world of Step Functions! They're fantastic, especially now with JSONata making automation so much easier. Regarding VPC access, I've heard that using EventBridge connections can help Step Functions interact with APIs in private/public subnets, but the setup can be pretty complex. A simpler VPC support, like what’s available with Lambda, would definitely make things easier. By the way, I’ve found that for any VPC needs, making a small Lambda that gets triggered by the State Machine works wonders.
Absolutely! Even with limited documentation, once you get familiar with Step Functions, they're incredibly powerful. I've processed massive data sets efficiently without breaking the bank.
Thanks for the insight! I think using Fargate to create a 'Data API' for Valkey is promising. It would be great if Step Function Express could run within a VPC to enhance its capabilities.