I'm looking for a reliable way to get the total number of blobs and their sizes in each container of my Azure storage account. The issue is that some containers have billions of blobs, so a straightforward list-blobs script isn't practical. Can anyone suggest an efficient method to automate this data retrieval on a daily or weekly basis without overwhelming my storage account? Ideally, I want to know the total blob count and total size (in GB or TB) per container, and the solution should scale well with large blob counts.
4 Answers
Just a heads up though, using Blob Inventory for billions of blobs can be really slow and costly. You'll rack up charges for storage and compute just to maintain those reports, and the data might be outdated by the time you get it.
Have you considered trying this preview tool from Microsoft? It's designed for storage discovery and might help with your needs.
You might want to check out the Blob Inventory feature. It's great for summarizing your data, and you can run calculations based on that. I used it before to create parquet files, and it gave me all the info I needed!
You could also explore using the Azure Graph API; it might offer some insights you're looking for.
Related Questions
How To Get Your Domain Unblocked From Facebook
How To Find A String In a Directory of Files Using Linux