I'm part of a small AI team using L40s on AWS, and our monthly costs are soaring over $3,000. We tried spot instances, but they aren't reliable for our workload. We're not ready to switch providers due to compliance and procurement issues, but the on-demand pricing is taking a toll. Has anyone discovered effective optimization strategies to help reduce costs while staying on AWS?
5 Answers
If you know which instance types you need, consider using reserved instances instead of on-demand. Prepaying for 12 months or more can offer great discounts.
Have you thought about reaching out to your AWS account manager? They might have some ideas or options to help reduce your costs.
Just curious, does 'L40s' refer to g6e instances? Also, do you power them down when they're not in use? If your workload can be spread out, maybe you can run parts on multiple smaller instances or use spot instances when they’re available. And are you already using a savings plan?
That's exactly what Savings Plans are for! If you can commit to a set monthly spending amount, you'll see significant savings compared to on-demand prices. Check it out here: https://aws.amazon.com/savingsplans/compute-pricing/.
Have you researched Savings Plans? They can really help you save if you're willing to commit to a specific monthly spend.
Related Questions
xAI Grok Token Calculator
DeepSeek Token Calculator
Google Gemini Token Calculator
Meta LLaMA Token Calculator
OpenAI Token Calculator