I'm a seasoned developer currently working on creating an AI marketplace. In this project, users can choose and compare results from various AI models across different applications such as image generation, text processing, and audio synthesis. However, I've run into legal issues while trying to use third-party providers like Replicate, even with open source models. To avoid complications, I've decided to eliminate third-party AI providers. Now, I'm looking for advice on where to host open source models for my application. What tech stack would you recommend? How can I optimize costs? Also, how do you suggest managing AI models so that they're only activated when requested, and how to handle the warming up process? Any insights would be greatly appreciated!
2 Answers
Are you considering LLMs or traditional ML models? If it’s LLMs along with more complex image and audio models, I would advise against self-hosting them. The computing resources needed today are massive, and your challenge will likely be budget and infrastructure rather than the hosting platform itself. On the bright side, Hugging Face offers some solid tutorials on deploying open source models, which can help you get started.
Just to clarify, I’m looking at LLMs too. When I say self-hosting, I actually mean running them on cloud platforms equipped with specialized hardware. Cost shouldn’t be a major issue since the revenue should cover it. I’m mainly searching for common patterns to help me rebuild this part of the application effectively. Thanks for the tip about Hugging Face, I'm hoping it will guide me on making things more cost-effective!
Related Questions
Biggest Problem With Suno AI Audio
How to Build a Custom GPT Journalist That Posts Directly to WordPress