The PaSh project, which develops tools for running shell scripts in parallel, has announced that it is coming under the auspices of the Linux Foundation, which will provide the infrastructure and services needed to continue development. The project code is distributed under the MIT license and includes components in Python, Shell, C, and OCaml.
PaSh includes JIT compiler, runtime and annotation library:
- Runtime provides a set of primitives to support the parallel execution of scripts.
- The annotation library defines a set of properties that describe situations in which individual POSIX and GNU Coreutils commands can be parallelized.
- The compiler on the fly parses the proposed Shell script into an abstract syntax tree (AST), splits it into fragments suitable for parallel execution, and forms a new version of the script based on them, parts of which can be executed simultaneously. Information about commands that allow parallelization is taken by the compiler from the annotation library. In the process of generating a parallel-executed version of the script, additional constructs from the Runtime are substituted into the code.
For example, a script that processes two files f1.md and f2.md cat f1.md f2.md | trAZ az | tr -cs A-Za-z '\n' | sort | uniq | comm -13 dict.txt -> out cat out | wc -l | sed 's/$/ mispelled words!/' would normally process two files in sequence:
Source: opennet.ru