Open-sourced tool speeds up Linux scripts via parallelization

The tool finds sections of code that can run independently, then runs them in parallel to save time.

car speedometer

Researchers have open-sourced (also called pash), a tool that can dramatically speed up Linux scripts by using parallelization, saving time and without risk of introducing errors.

The process of parallelization first examines a script for code that can be run separately and independently, so not all scripts can benefit from the tool. But when does find portions that can run independently, it runs them in parallel on separate CPUs. It also uses other techniques to get the code to run faster.

Below is a demonstration I ran on my home Fedora box, first running a script on its own and then again using Note that this script was provided with the tool and lends itself to parallelization. It’s not nearly as demanding as scripts that might process gigabytes of data in a scientific or artificial-intelligence lab, so the results are not dramatic.

Running the script on the command line

I used the time command to gauge the performance of the script.

$ time ./evaluation/intro/

real    0m55.077s
user    0m54.815s
sys     0m0.062s

NOTE: The "2176" on the second line is the script's output.

Running the script using

In the next command, I ran the same script through

$ time ./ ./evaluation/intro/

real    0m19.216s
user    0m37.509s
sys     0m0.255s

Notice that when run using, the script used little more than a third of the time (real time) that it used when run directly. If I run a script that simply loops from 1 to 10,000 and display the count every 100th step, it takes significantly longer to run using That’s because with, the script doesn’t benefit from parallelization but still requires an analysis:

$ time ./count_to_10000         $ time ./count_to_10000
100                             100
200                             200
300                             300
400                             400
500                             500
600                             600
700                             700
800                             800
900                             900
1000                            1000
real    0m0.010s                real    0m59.121s
user    0m0.007s                user    0m41.386s
sys     0m0.003s                sys     0m19.263s

The script runs a single loop and looks like this and provides no opportunity for parallelization:

for num in {1..1000}
  if [[ "$num" == *"00" ]]; then
    echo $num

For complex scripts that can benefit from parallelization, however, pash can make a tremendous difference in how long they take to run. All you have to do is invoke your scripts using And, as already noted, does this without introducing errors, so you can be confident that you will get the results expected, just a whole lot faster. If you are using scripts that need to process a large amount of data, this can save a lot of time.

Installing and using

You will need to have tools like sudo, wget, and curl, but these tools are likely already available on your Linux system.

Once is installed, you will need to export $PASH_TOP that will point to the top of the directory where it is installed. For example:

$ export PASH_TOP=/opt/pash
$ echo $PASH_TOP /opt/pash


From everything I've seen and read, can provide a dramatic performance improvement to complex and data-hungry scripts. If you or your organization might benefit from this kind of tool, it is well worth looking into. is hosted by the Linux Foundation and steered by a committee of researchers and practitioners that have been working on the system for more than two years.

The tool, as well as the example code, are open source, and is available at github. There is no man page, but help is available when you use the --help command. A technical paper explaining has been posted by Nikos Vasilakis, a research scientist at MIT’s Computer Science & Artificial Intelligence Laboratory (CSAIL) who chairs the committee working on the tool. MIT announced earlier this month. Stevens Institute of Technology is also involved in its development.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2022 IDG Communications, Inc.