PIPE&GREP(1)

NAME

pipe & grep

Notes from the Linux command line. A small manual of commands, written by one person, tested on a real machine.

SYNOPSIS

cmd1 | cmd2 | cmd3 [--read-the-output]

DESCRIPTION

Short notes about the Linux command line: how to put pipes together, what an awk or sed one-liner is doing, systemd timers against cron, ssh config, safe shell scripts and dotfiles. Each note explains the commands it uses and says where they differ between systems.

The example below is the whole idea in one picture. A command is cut into its stages, and each stage shows what it hands to the next.

Which addresses hit the server most? Five log lines, five stages.
access.log
  1. in< access.logthe file, 5 lines
    203.0.113.7 - - [07/Nov/2026:10:12:01] "GET / HTTP/1.1" 200
    198.51.100.4 - - [07/Nov/2026:10:12:03] "GET /a HTTP/1.1" 200
    203.0.113.7 - - [07/Nov/2026:10:12:09] "GET /b HTTP/1.1" 404
    192.0.2.88 - - [07/Nov/2026:10:12:15] "GET / HTTP/1.1" 200
    203.0.113.7 - - [07/Nov/2026:10:12:22] "GET /c HTTP/1.1" 200
  2. 1awk '{print $1}'keep field 1, the client address
    203.0.113.7
    198.51.100.4
    203.0.113.7
    192.0.2.88
    203.0.113.7
  3. 2sortput equal lines side by side
    192.0.2.88
    198.51.100.4
    203.0.113.7
    203.0.113.7
    203.0.113.7
  4. 3uniq -ccollapse neighbours, prefix a count
    1 192.0.2.88
    1 198.51.100.4
    3 203.0.113.7
  5. 4sort -rnbiggest count first
    3 203.0.113.7
    1 198.51.100.4
    1 192.0.2.88
  6. 5head -n 2stop after two lines
    3 203.0.113.7
    1 198.51.100.4

NOTES

Safer shell scripts, strict mode and quoting
What set -euo pipefail actually does, where it stays silent, how to quote properly, and a trap that cleans up after itself.
systemd timers or cron
A nightly backup written both ways, the units and the commands to manage them, and where each one is the better tool.
Five awk and sed one-liners, taken apart
Five GNU awk and sed one-liners worth memorising, each split into its tokens, plus the sed -i difference on macOS.
Build a pipeline one stage at a time
How to grow a shell pipeline stage by stage, with a web server log as the example, and why uniq needs sort first.

All notes

SEE ALSO

about(7), contact(7), privacy(7)