Right, let’s talk about sed. The name stands for “stream editor,” which sounds about as exciting as watching paint dry. But don’t be fooled. This is your text-processing power tool, your surgical instrument for slicing and dicing data on the command line. It’s the thing you’ll use to fix a thousand config files at once, extract specific bits of logs, or reformat data that some other program vomited out in a weird, sad format. Think of it as a super-charged, programmable “Find and Replace” that never gets tired and never asks for a raise.

Its core concept is beautifully simple: it reads input line by line, applies a series of commands you specify, and prints the result. The magic is in those commands.

The Workhorse: The Substitute Command (s/old/new/)

This is the command you’ll use 90% of the time. It looks like hieroglyphics at first, but you’ll learn to love it. The syntax is s/pattern/replacement/flags.

Let’s say you have a file names.txt:

Alice Smith
Bob Jones
Carol Williams

And you want to swap the first and last names. You’re not just replacing text; you’re capturing parts of it. This is where regular expressions and capture groups come in.

sed 's/^\([^ ]*\) \(.*\)$/\2, \1/' names.txt

Okay, breathe. Let’s break down that regex voodoo:

  • ^ and $ anchor the pattern to the start and end of the line.
  • \([^ ]*\) captures a group (\(...\)) of zero or more (*) non-space characters ([^ ]). This is the first name.
  • The space matches the, well, space.
  • \(.*\) captures another group of any characters (.*), which is the last name.
  • In the replacement, \2 refers to the second captured group (the last name), and \1 to the first. So we output “Last, First”.

The most common flag you’ll use is g, which stands for “global.” It replaces all occurrences on the line, not just the first one. Try sed 's/a/X/' vs. sed 's/a/X/g' on a line with multiple ‘a’s and you’ll see the difference instantly.

Printing and Deleting: p and d

By default, sed prints every line after it’s done processing it. The p command explicitly prints the current pattern space. If you use it without suppressing the default output, you’ll get duplicate lines. This is a classic rookie mistake.

The -n option tells sed, “Do not print anything unless I explicitly tell you to.” This turns p into a filter. Want to see only the lines that contain “error”?

sed -n '/error/p' /var/log/syslog

This is the poor man’s grep. It’s functionally identical to grep error, but it’s good to know how it works under the hood.

The d command is simpler: it deletes the current pattern space. sed immediately moves on to the next line without printing. Want to remove all comments (lines starting with #) from a config file?

sed '/^#/d' config.file

Poof. They’re gone.

Getting Precise with Ranges

Sometimes you don’t want to operate on every single line. You want to be specific. sed lets you define addresses—either line numbers or patterns—to limit where a command applies.

Want to delete lines 5 through 10?

sed '5,10d' myfile.txt

Want to perform a substitution only on lines 15 to the end of the file?

sed '15,$ s/foo/bar/g' myfile.txt

You can also use patterns. Want to delete everything from a line containing “START” until a line containing “END”?

sed '/START/,/END/d' myfile.txt

This is incredibly powerful for cutting blocks of text out of files. But here’s a pitfall: what if your file has multiple “START”/“END” blocks? The range will match the first “START” to the first “END”, then the next “START” to the next “END”, and so on. It doesn’t handle nested blocks gracefully because sed is line-oriented, not a parser for recursive structures. For that, you’d need something like awk or even a proper programming language.

Best Practices and Pitfalls

  1. Use Single-Quotes. Always. Your sed expressions will contain characters like <, >, $, and ` that your shell is desperate to interpret. Single-quotes protect them from the shell’s meddling hands. sed 's/$PATH/FOO/' will try to replace the literal string “$PATH”. sed "s/$PATH/FOO/" will replace it with the value of your environment variable, which is almost never what you want in a sed command.

  2. In-Place Editing: The -i Flag. This is sed’s “okay, get serious” mode. It tells sed to write the changes back to the original file. This is destructive. Always test your command without -i first to see the output on your screen. Once you’re sure it’s correct, use -i:

    sed -i 's/old/new/g' important_file.conf # For real this time.

    (Pro tip: sed -i.bak 's/old/new/g' file will create file.bak as a backup before modifying file. Do this.)

  3. The Separator Can Be Any Character. Using slashes s/one/two/ is convention, but if your pattern or replacement contains slashes, it becomes a mess of backslashes: s/\/usr\/local\/bin/\/usr\/bin/. Gross. Just use a different character, like a pipe or a colon: s|/usr/local/bin|/usr/bin|. Much cleaner. sed doesn’t care.

sed is a sharp tool. It rewards precision and punishes carelessness. Start simple, test relentlessly, and soon you’ll be wielding it like a master sculptor, not just someone swinging a sledgehammer.