How Can I Run Several Bash Commands in Parallel and Wait for Them to Finish?

0
1
Asked By CuriousCat92 On

I'm trying to execute a series of bash commands in a specific sequence. First, I need to run `php -f getCommands.php`, then I want to run two commands from `download.txt` and `update.txt` in parallel. After those finish, I need to run another `php` command (`getSecondSetOfCommands.php`), and then I want to execute two more commands from `download2.txt` and `update2.txt` in parallel. I've tried using `&` operator to run commands in the background, but it starts the next command prematurely. What's the correct way to manage this flow?

5 Answers

Answered By CommandMonkey On

To set this up, just make sure your commands are organized properly with waits in the right spots. Here’s a basic structure:
```bash
#!/usr/bin/bash

# First Command
php -f getCommands.php

# Parallel Tasks
{ sleep 10; } &
{ sleep 10; }
wait

# Follow-up Command
php -f getSecondSetOfCommands.php

# Final Parallel Tasks
{ sleep 6; }
{ sleep 6; }
wait
``` Make sure your syntax is correct, especially with the grouping for the parallel tasks. Using `wait` after the backgrounds ensures everything finishes as needed!

Answered By ScriptingSage On

Why not wrap your commands in a function and then call that function with a `wait` statement inside? It keeps the script organized and gives you better control over the execution flow!

Answered By TechieTurtle68 On

You can actually simplify that a lot using the `wait` command! Just run your first command normally, then start your parallel jobs with an ampersand after them but make sure to include a `wait` before the next command like this:
```bash
php -f getCommands.php
[ -f /tmp/download.txt ] && parallel -j4 --ungroup :::: /tmp/download.txt &
[ -f /tmp/update.txt ] && parallel -j4 --ungroup :::: /tmp/update.txt &
wait
php -f getSecondSetOfCommands.php
[ -f /tmp/download2.txt ] && parallel -j4 --ungroup :::: /tmp/download2.txt &
[ -f /tmp/update2.txt ] && parallel -j4 --ungroup :::: /tmp/update2.txt &
wait
``` This way, it will wait for all those parallel tasks to finish before moving on!

Answered By BashGuru91 On

Could `wait` work for your scenario? It’s designed to wait for background processes to finish. You can check more about it in details here: https://unix.stackexchange.com/questions/541311/bash-wait-for-all-subprocesses-of-script

Answered By SealOfApproval On

If you just want to combine files, you could consider something like `cat download.txt upload.txt > updown.txt`. It might be easier than running multiple `parallel` commands!

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.