I'm developing a web app that utilizes a heavy video-to-video ML model, focused on transformation and generation tasks. To keep my main API lightweight, I want to offload the processing to another service that can handle the heavy lifting. I was considering AWS SageMaker since it's designed for machine learning projects, but I've come across a lot of feedback suggesting it can be overpriced and somewhat cumbersome. On the other hand, setting up my own solution using ECS or EC2 seems like it could involve a lot of work to ensure proper scaling. Has anyone here had experience hosting similar models? Is SageMaker really worth it, or should I just roll out a container on ECS or EC2? Currently, my API is running on ECS with Fargate.
4 Answers
I would recommend having your API queue jobs in SQS and then use a Lambda function or Step Functions to process those messages and kick off ECS tasks. This setup can really streamline the process without heavy overhead.
Your current setup sounds fine to me! The main challenge seems to be that the lightweight API can't handle inferences directly, so offloading to a service is definitely smart. Just keep in mind that adding SageMaker might increase your costs by about 20%. That's worth considering if budget is tight.
Consider a Docker-based setup with Lambda functions for your processing. It's a cost-effective approach, especially if you don't need constant high availability, plus it simplifies scaling and queue management. I implemented a similar approach for a project with mass video and audio uploads, and it worked great using ffmpeg for conversion.
If you're looking for a low-cost solution, check out beam.cloud. They offer a serverless way to run video-to-video pipelines with a solid user base. It's easy to use; you can deploy your ML code with just a few lines of code!
For this kind of project, though, I might need some GPUs to really handle the load.