I'm curious about managing EBS volumes automatically. We know it's tough to shrink or grow block storage volumes, whether on-premises or in the cloud, especially given that EC2 has its limitations. Autoscaling tends to work only in one direction, which is frustrating. I found three vendors offering solutions for automated EBS volume management: Datafy, ZestyDisk, and Lucidity. Has anyone tried these or have insights on how to address volume management beyond the usual method of copying data to smaller volumes? I'm aware of FSxN's deduplication and thin provisioning capabilities, but I'm not looking to go in that direction. What strategies are people using to manage storage effectively?
4 Answers
Shrinking EBS volumes can be quite complicated. Many teams end up over-provisioning just to avoid the hassle or rely on snapshots and recreating volumes during planned downtimes. I've learned to evaluate how these tools handle rollbacks and sudden I/O increases before fully trusting any automation. Interestingly, in some cases I've noticed significant savings by identifying dev/test instances that were running 24/7 without need, so scheduling non-production infrastructure to shut down during off-hours often has a bigger impact than merely resizing.
It really depends on your needs. Resizing EBS volumes isn't the end goal; there's usually a larger issue at play. If you require online resizing, I'd recommend sticking with something like LVM where you can add EBS physical volumes as needed. Just use pvmove and detach when you need to adjust things.
If your application can tolerate slower access times, consider AWS Storage Gateway. It uses S3 as storage while offering SMB or NFS shares to your network. It could reduce costs since you would only worry about the gateway instance and S3 storage fees, which are based on consumption. Sometimes, using S3 and dealing with its I/O can be cheaper than sticking with EBS, depending on your setup.
EBS prices are quite affordable, so it's usually better to just provision what you need and scale up slowly. If you ever need to shrink a volume, the process involves copying your data to a new volume and switching them out. Currently, there's no reliable way to just decrease the size of an existing volume directly.
I get that, but honestly, the time spent figuring out the right tool or paying for a solution often doesn't match the benefits. Most of the time, you can scale your storage as needed without worrying about shrinking it later. I've rarely seen scenarios requiring significant space reduction.