I recently migrated a large amount of data from my local storage to Amazon S3, and I've noticed that the total size reported in S3 is consistently smaller than what I had on-premises, even though all files were uploaded successfully. I'm wondering if this size difference is due to S3's storage architecture, or is there another reason behind it?
3 Answers
If you're comparing to the "size on disk" that most local storage tools report, that might change things. This can be impacted by sector size, especially with a lot of small files or if your local storage has larger sectors. S3 bills you for the actual size without rounding, so that could be a reason for the discrepancy.
Have you checked if the size difference is due to the conversion between GB and GiB? S3 reports in GiB, so that could mess up your calculations if you're mixing them up. Just a thought!
When I look at S3 storage usage, it shows it in GB, not GiB. Can you explain why you think S3 uses GiB?
You might not be considering the block size of your local file system. This can also lead to differences in reported sizes.
Exactly, sector size plays a significant role here. Those who haven’t dealt with storage admin tasks or RAID setups might not catch that right away!