Discussion: standard UNIX utilities that few people have used and still use today

A week ago, Douglas McIlroy, the developer of the UNIX pipeline and the author of the concept of 'component-oriented programming', explained talked about interesting and unusual UNIX programs that have not found widespread application. The publication sparked active discussion on Hacker News. We have gathered the most interesting insights and would be happy if you join the conversation.

Photo - Virginia Johnson - Unsplash

Working with Text

In UNIX-like operating systems, there is a standard set of tools for text formatting. The utility typo allowed users to check documents for typos and hapaxes — words that occur only once in the material. Interestingly, to find typos, the program does not use uses dictionaries. It relies solely on the information in the file and conducts a frequency analysis of trigrams (sequences of three characters). Meanwhile, all necessary counters are stored in a 26x26x26 array. According to Douglas McIlroy, such memory was barely enough for a few single-byte counters. Therefore, to save space, they were written in logarithmic form.

Today, typo has been replaced by more modern and accurate spell-checking programs based on dictionaries. However, the tool is still remembered — a few years ago, an enthusiast presented implemented typo in Go. The repository is still being updated.

Another tool for document management from the '80s is the Writer’s Workbench by Lorinda Cherry and Nina McDonald from Bell Labs. It included tools for identifying parts of speech and the style of a document, finding tautologies, and overly complex sentences. The utilities were developed as support for students, and at one time their usage was widespread among students at the Colorado State University in the USA. However, by the early '90s, Writer’s Workbench was forgotten because it was not included in Version 7 Unix. Nevertheless, this tool continued to influence imitations — for example, they used Grammatik for IBM PC. In UNIX, there are also standard tools that simplify working with formulas. There is a language preprocessor for formatting mathematical expressions

. It is noteworthy that to display a formula, the developer only needs to describe it using simple words and symbols. Keywords allow adjusting mathematical signs vertically and horizontally, changing their sizes and other parameters. If you pass the utility a string: eqn. It is notable that for displaying the formula, the developer only needs to describe it using simple words and symbols. Keywords allow for shifting mathematical signs vertically and horizontally, adjusting their sizes and other parameters. If you pass the utility a string:

sum from { k = 1 } to N { k sup 2 }

The following formula will be generated:

In the 1980s–1990s eqn helped IT specialists write manuals for software. But later it was replaced by the LaTeX system, which use even Habr. But eqn is the first tool of its kind that remains part of UNIX-like OS.

File handling

In a thematic thread, Hacker News residents noted several rarely used utilities for file handling. One of them is a comm for their comparison. It is a simplified version diff, designed for use in scripts. Its wrote creator Richard Stallman along with David MacKenzie.

The program output consists of three columns. The first column contains values unique to the first file, the second contains values unique to the second file. The third column includes common values. For comm to work correctly, the files being compared must be lexically sorted. Therefore, one resident of the site noted that proposed to work with the utility in the following way:

comm <(sort fileA.txt) <(sort fileB.txt)

Comm is convenient for verifying spelling of words. Just compare them with a reference dictionary document. Given the nuances related to the necessity of sorting files, there are an opinion, which Stallman and MacKenzie wrote their utility specifically for this use case.

Photo - Marnix Hogendoorn - Unsplash

Also a participant in the discussion on HN noted the capabilities of the paste, which were not obvious to him. It allows interleaving data streams or splitting one stream into two columns during output:

$ paste <( echo -e 'foonbar' ) <( echo -e 'baznqux' )
foo     baz
bar     qux
$ echo -e 'foonbarnbaznqux' | paste - -
foo     bar
baz     qux

One user noticed, that often, to perform these simple operations, less optimal solutions are used: starting with fmt, ex and ending with mlr with the ability to output an infinite stream of random data within specified boundaries has been added; and rs.

What standard capabilities of UNIX-like operating systems have been a revelation for you?

What we write about in our corporate blog:

How the domain name system evolved: the ARPANET era
The History of the Domain Name System: The First DNS Servers
The history of DNS: when domain names became paid
The history of the domain name system: 'protocol wars'

Source: habr.com