I'm looking to transfer a massive 100 TB of uncompressed files (around 20 TB when compressed) over a WAN connection with speed capabilities of about 5 Gbps. However, I'm in a bit of a bind because port 22 is blocked, and the network security team has made it clear that they won't allow any exceptions for SSH or SFTP. Normally, I would use rsync over SSH for this kind of task, but I'm wondering if there's a way to make rsync work on a different port without needing to configure SSH for that port. What alternatives do I have that are either already part of RHEL or can be included? Any help would be appreciated!
5 Answers
Instead of trying to bypass network security rules, I suggest reaching out to the security team directly. They should have a set process for approved file transfers. Given it’s a data center migration, they might already have established methods like VPN or secure file transfer solutions in place to help with your requirement.
Have you thought about setting up a temporary VPN between your two datacenters? This would allow you to use port 22 without exposing it to the external network, and keep the transfer secure. There are also vendor solutions like MoveIt, if they allow it, which handle large file transfers efficiently.
Good point about the VPN! It's often the best way to handle such transfers. Just ensure everyone is on board with the security protocols required.
Yes, you can run SSH on a different port if you have the appropriate permissions. Even though port 22 might be blocked between datacenters, you can configure SSH to listen on a non-standard port and use that with rsync. Just make sure you set the `RSYNC_RSH` environment variable or use `-e 'ssh -p your_port_number'` in your rsync command to specify that port. That way, you can avoid conflicts with the firewalls.
Exactly! Just make sure whatever port you choose is open on both sides for the transfer to work smoothly.
If all else fails, just ask the network team for a specific port if they allow some exceptions. Or use rsync in daemon mode on a port other than 22 if that's an option. Good luck!
Super helpful! Rsync can be a lifesaver if configured properly.
If you want a more physical approach, consider shipping hard drives overnight. AWS even allows you to send hard drives for large transfers. A little humor here, but a shipping container ship has been nicknamed the ultimate data transfer method—cheap and effective!
No kidding! I read about similar RFCs where the fastest data transfers come from shipping drives across oceans! The postal service is a surprisingly viable option.
Absolutely, it's better to clarify what methods are approved rather than risk running afoul of compliance. Just be sure to explain the whole situation and timelines to your security team.