Sed command: tips and tricks

I decided a year ago to learn about sed and awk: two powerful command line based text processors. You can do really awesome tricks using them. Especially when you're writing a shell script, including sed and/or awk may add bits of awesomeness to your script. You can for instance filter white spaces, parse strings in a defined way, etc..

But today, i'm going to show you what i have learned so far! it sounds creepy, but it's not that hard to learn. Let's Stream EDit!

Substitution

Substitute "foo" with "bar" on each line
Only the 1st occurrence

sed 's / foo / bar /'

For the 3rd case only

sed 's/foo/bar/3'

All occurrences

sed 's / foo / bar / g'

Just before the last occurrence

sed 's / \ (.* \) foo \ (.* foo \) / \ 1bar \ 2 /'

Only the last case

sed 's / \ (.* \) foo / \ 1bar /'

Substitute "foo" with "bar" only lines containing "plop"

sed '/ plop / s / foo / bar / g'

Substitute "foo" with "bar" except the lines containing "plop"

sed '/ plop /! s / foo / bar / g '

Replace "Foo" or "foo" with "bar" on each line

sed 's / [Ff] oo / bar / g'

Replace "blue" or "white" or "red" with "green"

sed 's / blue \ | blank \ | red / green / g'

Display

The 1st line (head -1)

sed q

The first 5 lines (head -5)

sed '5 q '
sed '1, 5! d '

The last line (tail -1)

sed -n '$ p'
sed '$! d "

The last 5 lines (tail -5)

sed -e: a-e '$ q, N, 6, $ D; ba'

The 2 last lines (tail -2)

sed '$! N; $! D'

Only lines match a a reason or a regular expression

sed -n '/ pattern / p'
sed '/ regexp /! d'

Only lines that match a not a reason or a regular expression

sed -n '/ pattern /! p'
sed '/ regexp / d'

The line preceding a pattern or a regular expression

sed -n '/ pattern / (g; 1! p;) h'

The line following a pattern or a regular expression

sed -n '/ regexp / (n, p;)'

Remove

Removing spaces and tabs
At the beginning of the line

sed 's / ^ [\ t] * / / 
sed 's / ^ \ s * / /' # Using the parameter "\ s"

At end of line

sed 's / [\ t ]*$//'

At the beginning and end of line

sed 's / ^ [\ t ]*//; s / [\ t ]*$//'

Removing blank lines
All empty lines

sed '/^$/ of 
sed '/./! of

Only those at the top

sed '/./,$! of 
sed -nr'/./,$ /(.*)/ s \ 1 / p '

Only those at end

sed -e: a-e '/ ^ \ n * $ / ($ d N; ba'-e ')'

Eliminate a line at regular intervals
All lines pairs

sed '1 ~ 2d '

All the odd lines

sed '2 ~ 2d '

Every n lines from the line n

sed '3 ~ 2d '# 2 All lines from line 3

Miscellaneous

Join lines
Attach lines 2 by 2

sed '$! N s / \ n / /'

Attach the 3 lines by 3

sed '$! N s / \ n //;$! N s / \ n / /;'

If a line ends with a backslash (\), add the following line and replace the end of line (\ n) by a space

sed -e: a-e '/ \ \ $ / N s / \ \ \ n / /; ta'

If a line begins with an equal sign (=), add it to the previous line and replace the equal sign (=) with a space

sed -e: a-e '$! N s / \ n = / /; ta'-e 'P, D'

there are also so many other uses and techniques for sed that we cannot cover all in one post, maybe you know some, share them with us in the comment box below!