Hey everyone! I'm looking for advice on cost optimization for a client that has over 50 AWS accounts, specifically when dealing with EBS/EC2 snapshots that are 3 to 5 years old. How do you approach this?
1. Is there a generally accepted cutoff period where we could start to think about what to keep or discard, say after 3 years?
2. Should we have a standard practice of retaining only the most recent 5 snapshots?
I think it's crucial to determine some guidelines here, whether we propose our recommendations to the client or seek their input on what they prefer for managing old snapshots.
I've explored AWS Cost Explorer, but it seems tricky to get detailed insights into the costs. Plus, manually sorting through these accounts isn't feasible.
I'd love to hear how others have approached similar challenges. Any insights would be greatly appreciated! Thanks in advance!
5 Answers
Manual checking isn’t the way to go. AWS is API-driven, so writing a script in Python or another language to list all snapshots across accounts and export to CSV is really doable. It'll give you a first pass overview, and I'm sure your client would appreciate having that data to start discussions.
You should also consider the Recovery Time Objective (RTO) and Recovery Point Objective (RPO) for their operations. I've set mine to 72 hours and 24 hours, hence I keep a week’s worth of AMIs and snapshots. It might help guide your retention policies too!
I suggest you set a 90-day cut-off specifically for development snapshots. It's easy for people to create snapshots and forget about them. Just ensure you get agreement with everyone involved before you put a plan into action. For production snapshots, keep a clear retention strategy and consider moving persistent data over to S3.
Using AWS Backup can help automate keeping the last few snapshots for a defined number of days. I really feel that after about a month, the data's usefulness deteriorates due to changes in the app or system. If your client is looking into long-term storage, encourage them to use S3 with lifecycle rules to transition data to Glacier for lower costs.
Thanks for your insight! I think they might be using Veeam, but I’ll investigate the current backup strategy. Appreciate your reply!
Really, the best direction here should come from your client. No one knows their specific needs like they do. Have a chat with them about how often they actually restore from those old snapshots and whether they really need to keep them for that long. Personally, I would recommend a solid retention cut-off, like keeping just 90 days' worth. This could significantly reduce costs and allow them to focus on other operations that really require long-term backup.
Just a heads up, in financial services, they often need to keep records for 7+ years. So it’s best to confirm their requirements before making assumptions.
Thanks for your reply! That makes sense. Have you got any tips or methods for consolidating all the EBS/EC2 snapshots into a report that the client can easily digest?
Thanks! Since I'm new here, I'll need to confirm these metrics. Understanding them will definitely help in crafting my suggestions going forward.