Command Pipelines

When writing bash scripts, I often connect a series of commands together into a pipeline, where the commands are separated by the pipe | character.

When using pipelines, the output from the first command is treated as the input to the second command, the output of the second command is treated as the input to the third command, and so on.

This can be useful in a number of situations, such as when you need to process the output of a command further before displaying or assigning to a variable.

For example, given a file containing a sequence of numbers

$ cat numbers.txt
2250
2262
1
1
1
15379
15379
1
16112
16121

We can find the numbers in the file with the largest distribution as follows

$ sort -n numbers.txt | \
        uniq -c | \
        sort -rn | \
        head
141 2
 69 1685
 59 1
 53 2950
 11 1902
  4 2870
  4 2132
  3 9151
  3 4345
  3 1796

Where we first sort the contents of the file, using -n to sort them numerically, then pipe that output into the uniq command with the -c option to count the unique values, then sort again, this time with -rn for reverse numeric order, and finally, take the first 10 entries in the output (10 is the default number of lines that head will return.)