Hey everyone, I'm looking for the best way to automatically sync files from one directory to another machine whenever there's a change. Let me break down my situation:
- I have a program running on Machine A that writes files to a directory, either one file per hour or even one file per second depending on certain events.
- Machine B is located at another site and should receive these files from Machine A with the least possible delay.
So far, I've come up with a few solutions, but I'm not too thrilled about them:
1. Mounting an NFS - The downside is that if the connection to Machine B goes down, the program on Machine A crashes because it can't write.
2. A cron job with rsync that runs every minute - It's not the worst, but it's definitely not ideal.
3. A basic bash script that watches for changes and triggers rsync - This feels a bit better, but I'm wondering if there's something more efficient and less embarrassing to explain to others. Any ideas?
2 Answers
Have you thought about using inotify? It's a Linux utility that lets you monitor filesystem events. You can set it up to watch the directory on Machine A and trigger rsync whenever a file is written. It's usually more responsive than cron, plus it's pretty straightforward once you get the hang of it! Just keep in mind that if it's a very active directory, you might run into performance issues.
If you're looking for a more polished solution, lsyncd might be worth checking out. It's a live syncing daemon that uses inotify under the hood. You just configure it to monitor your source directory and point it to the destination. Should be less embarrassing when explaining your setup! Plus, it handles dropped connections gracefully.
That's true! Inotify can be really efficient, but I'd recommend testing it first. If your directory sees a lot of changes, it could overwhelm the event queue.