This chapter covers the
-f command line options. These come in handy for specific use cases. For example, the
-z option helps to process input separated by ASCII NUL character and the
-f option allows you to pass
sed commands from a file.
-z option will cause
sed to separate lines based on the ASCII NUL character instead of the newline character. Just like normal newline based processing, the NUL character is removed (if present) from the input line and added back accordingly when the processed line is printed.
NUL separation is very useful to process filenames as newline is a valid character for filenames but NUL character isn't. For example,
find -print0will print NUL separated filenames whereas
xargs -0will accept NUL separated input.
$ printf 'earn xerox\0at\nmare\npart\0learn eye\n' | sed -nz '/are/p' at mare part $ # \0 at end of output depends on input having \0 character $ printf 'earn xerox\0at\nmare\npart\0learn eye\n' | sed -nz '/are/p' | od -c 0000000 a t \n m a r e \n p a r t \0 0000015 $ printf 'earn xerox\0at\nmare\npart\0learn eye\n' | sed -nz '/eye/p' | od -c 0000000 l e a r n e y e \n 0000012
If input doesn't have NUL characters, then
-z option is handy to process the entire input as a single string. This is effective only for files small enough to fit your available machine memory. It would also depend on the regular expression, as some patterns have exponential relationship with respect to data size. As input doesn't have NUL character, output also wouldn't have NUL character, unlike
GNU grep which always adds the line separator to the output.
$ # adds ; to previous line if current line starts with c $ printf 'cater\ndog\ncoat\ncutter\nmat\n' | sed -z 's/\nc/;&/g' cater dog; coat; cutter mat
See my blog post Multiline fixed string search and replace for more examples with
-s option will cause
sed to treat multiple input files separately instead of treating them as single concatenated input. This helps if you need line number addressing to be effective for each input file.
-ioption is being used,
-sis also implied.
$ # without -s, there is only one first line for all files combined $ # F command inserts filename of current file at the given address $ sed '1F' cols.txt 5.txt cols.txt 1:2:3:4 a:b:c:d five 1five $ # with -s, each file has its own address $ # this is like 'cat' command but also prints filename as header $ sed -s '1F' cols.txt 5.txt cols.txt 1:2:3:4 a:b:c:d 5.txt five 1five
sed commands does not easily fit the command line, you have the option of putting the commands in a file and use
-f option to specify that file as the source of commands to execute. This method also provides benefits like:
- literal newline can be used to separate the commands
- single quotes can be freely used as it will no longer clash as a shell metacharacter
- comments can be specified after the
- See also sed manual: Often-Used Commands for details about using comments
Consider the following input file and a
sed script written into a plain text file.
$ cat sample.txt Hi there cats and dogs running around Have a nice day $ cat word_mapping.sed # word mappings s/cat/Cat/g s/dog/Dog/g s/Hi/Hey/g # appending another line /there|running/ s/$/\n----------/
The two lines starting with
# character are comment lines. Comments can also be added at end of a command line if required, just like other programming languages. Use the
-f option to pass the contents of the file as
sed commands. Any other command line option like
-E, etc have to mentioned along with
sed invocation, just like you've done so far.
$ # the first 3 substitutions will work $ # but not the last one, as | is not a metacharacter with BRE $ sed -f word_mapping.sed sample.txt Hey there Cats and Dogs running around Have a nice day $ # | now works as ERE is enabled using -E option $ sed -E -f word_mapping.sed sample.txt Hey there ---------- Cats and Dogs running around ---------- Have a nice day
Similar to making an executable
python script, you can add a shebang (see wikipedia: shebang for details) line to a
sed script file.
$ # to get full path of the command $ type sed sed is /usr/local/bin/sed $ # sed script with shebang, note the use of -f after command path $ cat executable.sed #!/usr/local/bin/sed -f s/cats\|dogs/'&'/g $ # add execute permission to the script $ chmod +x executable.sed $ # executing the script $ ./executable.sed sample.txt Hi there 'cats' and 'dogs' running around Have a nice day
Adding any other command line option like
-E, etc depends on a lot of factors. See stackoverflow: usage of options along with shebang for details.
|change line separator from newline to ASCII NUL character|
|contain NUL characters, assuming small enough input file to fit memory|
|separate addressing for multiple file inputs|
|supply commands from a file|
|literal newline can be used to separate the commands|
|single quotes can be freely used|
|comments can be specified after the |
|command to insert current filename at the given address|
This chapter covered three command line options that come in handy for specific situations. You also saw a few examples of
sed being used as part of a solution with other commands in a pipeline or a shell script. In the next chapter, you'll learn three commands that are also specialized for particular use cases.
a) Replace any character other than word characters and
. character with
_ character for the sample filenames shown below.
$ mkdir test_dir && cd $_ $ touch 'file with spaces.txt' $'weird$ch\nars.txt' '!firstname.lastname@example.org' $ # > at start of line indicates continuation of multiline shell command $ for file in *; do > new_name=$(printf '%s' "$file" | sed ##### add your solution here) > mv "$file" "$new_name" > done $ ls file_with_spaces.txt _f_oo.txt weird_ch_ars.txt $ cd .. && rm -r test_dir
b) Print only the third line, if any, from these input files:
$ sed ##### add your solution here This game is good project you always wanted bla bla bla
c) For the input file
hex.txt, use content from
replace.txt to perform search and replace operations. Each line in
replace.txt starts with the search term, followed by a space and then followed by the replace term. Assume that these terms do not contain any
$ cat hex.txt start address: 0xA0, func1 address: 0xA0 end address: 0xFF, func2 address: 0xB0 $ cat replace.txt 0xA0 0x5000 0xB0 0x6000 0xFF 0x7000 $ sed -f <(sed ##### add your solution here) hex.txt start address: 0x5000, func1 address: 0x5000 end address: 0x7000, func2 address: 0x6000