I'm dealing with a substantial Azure Files situation and I'm looking for some advice. I have a single storage account that hosts a number of shares, with sizes ranging from 1GB to 15TB. All of this data is currently on the Transaction Optimized tier. We also have VNet grants in place, and the VM we're using for the migration has the Microsoft.Storage.Global SEP applied, plus a firewall exists to ensure security.
The reason for this copy is that we need to move our Azure Files workload to a different region since our current region is 'full' for compute resources. The target storage account is Azure Files Provisioned v2, as it offers significant cost savings. Unfortunately, the paired region lacks Availability Zones, so I'm opting for the next best region that does have them.
I initially tried using AzCopy for the migration, as it was touted to leverage 'Server to Server APIs' for better performance, but I've encountered major slowdowns. My files are mostly small documents and images, with the larger shares containing millions of files. Even with maxed out concurrency settings, the best I've managed is around 5MBps on a test workload of smaller files. Since this is such a large undertaking—120TB in total—I'm wondering if anyone has any suggestions or alternative methods to achieve acceptable speeds for this migration.
3 Answers
I’d recommend posting the specific AzCopy command you’re using; sometimes the syntax can impact speed greatly. Also, there's a thread that might be helpful regarding Azure Files migration; you should check it out.
With that amount of data, have you thought about reaching out to Microsoft to see if they can assist with the transfer? Alternatively, you could utilize Azure Data Box to download your data and move it physically. I’ve seen significantly higher rates when not relying on Azure Files specifically, which can be quite slow.
Have you considered using cross-region restore? If your Azure File Shares are backed up and you have cross-region restore enabled, you could restore your data in the target region and then run Robocopy to keep the data in sync until you're ready to switch over. Just keep in mind that restoring 15TB could take some time.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures