I'm trying to optimize my live migrations on a Proxmox PVE host. Right now, I have a one-liner that handles migration for all VMs on the host, except for the one with VMID 120. Given that I have 2 NICs in adaptive-alb, I want to take advantage of that by running 2 migrations simultaneously to speed things up. What changes can I make to my one-liner to achieve this? Ideally, I want it to handle 2 migrations at a time, and as soon as one finishes, it starts another. Is this possible? I was thinking of adding a nested loop to count the number of ongoing processes and wait if they reach 2 before continuing. Here's my current command:
```bash
time for vmid in $(qm list | awk '$3=="running" && $1!="120" { print $1 }'); do qm migrate $vmid pve3 --online --migration_network 10.100.80.0/24 --bwlimit 400000; done
```
4 Answers
Definitely consider using GNU Parallel. It’s great for running tasks concurrently and can simplify your command significantly.
You could use `xargs` to run your migrations in parallel. If you have a list of commands, you could use something like: `cat listOfCommands | xargs -I % -P2 %` to manage 2 processes at a time. This way, you won't need a traditional loop to handle the migrations.
Remember to think about the potential failures. When you run migrations in parallel, if something goes wrong, it can be difficult to determine exactly what failed. Just something to keep in mind when designing your solution!
If you're on bash 4.3 or later, here's a simple way to allow for parallel processes:
```bash
n=2 # number of parallel processes
i=0
qm list | awk ... | while read -r vmid ; do
if (( i++ >= n )); then
wait -n
fi
qm migrate "$vmid" ... &
done
```
This lets you run multiple migrations at once without blocking the rest.
That makes sense! Using `wait -n` is a smart way to handle the parallelization. Thanks for the tip!
Nice! This should really help with the migration speeds.