Since I’m new to Linux, apologies if I’m not using the correct terminology.
I’m trying to run a number of tasks in parallel (using the background shell operator). However, I need to first run two tasks in parallel followed (only after completion) by the rest of the tasks that depend on the outcome of the first two.
For example, suppose tasks A.1
and A.2
can run in parallel (no dependency between them) and produce some files that will be used by tasks B.1
, B.2
, and B.3
. All the B
tasks are independent; however, they cannot start until A.1
and A.2
finish in order to see the files. With this in mind, I came up with the following shell scripts:
script1
has the following lines:
./A.1 & ./A.2 &
script2
has the following lines:
./B.1 & ./B.2 & ./B.3 &
script3
simply calls scripts 1 and 2 sequentially (i.e., without the &
operator) as follows:
./script1 ./script2
I’m not sure if script3
is what I’m looking for. Am I doing it right? In other words, will script2
wait until script1
finishes and then starts?
Answer
No, that won’t do what you want. But it’s actually simpler than that.
First of all, at a simple level, your analysis is correct.
If script3
says
./script1 ./script2
then, yes, script3
will wait for script1
to finish,
and then it will start script2
.
The problem is that script1
will “finish” essentially immediately —
i.e., in the amount of time it takes to spawn the two A
processes —
and so the B
commands will be started
while the A
commands are still running. Not what you want.
The solution is
./A.1 & ./A.2 & wait ./B.1 & ./B.2 & ./B.3 &
The wait
command (as its name suggests)
waits for the existing asynchronous processes to terminate before it proceeds.
The wait
command must be executed by the same shell
that spawned the A
processes. You could put all six commands
into a single script, or the first three in one and the last three in another,
or various other combinations.
Also, you might want to do a wait
after the B
commands, too.