I'm in the process of deploying a containerized application on AWS that will have over 1000 pods. Some of these pods will need access to shared storage for file management. I've considered using Amazon EFS, but I've found that it becomes quite expensive at this scale. I'm looking for alternative solutions that provide a balance between cost and performance. I'm also open to creative setups or self-managed options, so any suggestions would be appreciated!
5 Answers
It might help to dig a bit deeper into your file management needs. What’s the size of these files? How often are they accessed or modified? This will influence your storage choice. For instance, if you need things cached or have multiple writers, that can change the game completely.
We've tackled a similar issue with a huge file download from S3. Instead of downloading everything during initialization, we set up a DaemonSet to sync the files to a local path. This approach enhanced our app's startup time significantly by removing the need for restarts to grab the latest files from S3.
Have you thought about refactoring your application? If you can eliminate cluster-wide persistent storage, that could save you a lot in costs. Sharing storage tends to be expensive and can slow down performance.
Re-architecting sounds like a solid option, but I understand it can be tough if you don't have control over the application layer.
Storing files directly in S3 can be an option, but be cautious of the latency. I looked into it, but the speed wasn't practical for my use case.
Yeah, that latency can be a deal breaker for active apps needing fast access.
Consider using Amazon EBS Multi-Attach. It’s high-performing and can be more cost-effective than EFS. However, setting up EBS as a cluster-aware filesystem can be tricky. If you find good tools for that, let me know!
Have you looked into OCFS2 or GFS2? They can handle these types of setups.
Great point! The files vary from 1MB to 4GB, and since this is for a collaborative tool with multiple users accessing and modifying files, we really need a robust solution.