The PaSh project, which develops tools for parallel execution of shell scripts, has announced its transition under the auspices of the Linux Foundation, which will provide the infrastructure and services necessary for continued development. The project's code is distributed under the MIT license and includes components in Python, Shell, C, and OCaml.
PaSh includes a JIT compiler, runtime, and an annotation library:
- The runtime provides a set of primitives to support parallel execution of scripts.
- The annotation library defines a set of properties that describe situations where the parallelization of individual POSIX and GNU Coreutils commands is permissible.
- The compiler on the fly parses the provided shell script into an abstract syntax tree (AST), breaks it into fragments suitable for parallel execution, and forms a new version of the script based on them, where parts can be executed simultaneously. Information about commands that allow parallelization is taken by the compiler from the annotation library. During the generation of the parallel executable version of the script, additional constructs from the runtime are inserted into the code.
For example, a script that processes two files f1.md and f2.md cat f1.md f2.md | tr A-Z a-z | tr -cs A-Za-z ‘\n’ | sort | uniq | comm -13 dict.txt — > out cat out | wc -l | sed ‘s/$/ mispelled words!/' in normal circumstances would process the two files sequentially:
Source: opennet.ru
