NAME
pipe & grep
Notes from the Linux command line. A small manual of commands, written by one person, tested on a real machine.
SYNOPSIS
cmd1 | cmd2 | cmd3 [--read-the-output]
DESCRIPTION
Short notes about the Linux command line: how to put pipes together, what an awk or sed one-liner is doing, systemd timers against cron, ssh config, safe shell scripts and dotfiles. Each note explains the commands it uses and says where they differ between systems.
The example below is the whole idea in one picture. A command is cut into its stages, and each stage shows what it hands to the next.
access.log
- in
< access.logthe file, 5 lines203.0.113.7 - - [07/Nov/2026:10:12:01] "GET / HTTP/1.1" 200 198.51.100.4 - - [07/Nov/2026:10:12:03] "GET /a HTTP/1.1" 200 203.0.113.7 - - [07/Nov/2026:10:12:09] "GET /b HTTP/1.1" 404 192.0.2.88 - - [07/Nov/2026:10:12:15] "GET / HTTP/1.1" 200 203.0.113.7 - - [07/Nov/2026:10:12:22] "GET /c HTTP/1.1" 200 - 1
awk '{print $1}'keep field 1, the client address203.0.113.7 198.51.100.4 203.0.113.7 192.0.2.88 203.0.113.7 - 2
sortput equal lines side by side192.0.2.88 198.51.100.4 203.0.113.7 203.0.113.7 203.0.113.7 - 3
uniq -ccollapse neighbours, prefix a count1 192.0.2.88 1 198.51.100.4 3 203.0.113.7 - 4
sort -rnbiggest count first3 203.0.113.7 1 198.51.100.4 1 192.0.2.88 - 5
head -n 2stop after two lines3 203.0.113.7 1 198.51.100.4
NOTES
- Safer shell scripts, strict mode and quoting
- What set -euo pipefail actually does, where it stays silent, how to quote properly, and a trap that cleans up after itself.
- systemd timers or cron
- A nightly backup written both ways, the units and the commands to manage them, and where each one is the better tool.
- Five awk and sed one-liners, taken apart
- Five GNU awk and sed one-liners worth memorising, each split into its tokens, plus the sed -i difference on macOS.
- Build a pipeline one stage at a time
- How to grow a shell pipeline stage by stage, with a web server log as the example, and why uniq needs sort first.