I have around 15TB of data stored in an Azure file share and need to move it to another subscription located in a different region. I'm concerned about the reliability of the transfer, as handling such a large volume of data might lead to failures. Would AzCopy be a good option for this task? I'm worried that running it from my laptop could lead to issues with interruptions or other problems.
5 Answers
I had a rough experience with AzCopy when moving data; depending on the type of files, it can be quite inefficient. I switched to Robocopy for handling smaller files and it significantly sped things up. Just a heads-up if you run into issues with AzCopy!
We used Azure Recovery Services vault for a similar move. Just keep in mind that there are traffic costs. It worked well as part of our backup strategy.
Running AzCopy from your laptop is a bad idea; your ISP can be a bottleneck. Instead, spin up a VM in the source Azure region to handle it. This way, you'll benefit from better speeds and AzCopy's ability to resume stalled transfers.
Consider using Azure Storage Mover for this task. It's designed for large data transfers and can simplify the process. Just make sure to plan for any associated costs.
If your data has multiple top-level folders, you can run several instances of AzCopy simultaneously. Since it utilizes Azure's internal network, you should get decent speeds. We managed a similar transfer of about a petabyte, and while it took weeks, AzCopy did the job well overall.
That's a great idea! Also, you could just set up a small VM in Azure to run AzCopy instead of using your laptop. It avoids ISP issues and interruptions.