Adding content from file

The previous chapter discussed how to use a, c and i commands to append, change or insert the given string for matching addresses. Any \ in the string argument is treated according to sed escape sequence rules and literal newline characters are not allowed.

The r and R commands use file content as the source string, which is always treated literally and can contain newline characters. Thus, these two commands provide a robust way to add text literally. However, these commands provide only the append functionality. Other sed features will be used to construct change and insert variations.

The example_files directory has all the files used in the examples.

Append entire file using r

The r command accepts a filename argument and the entire content of the given file is added after each of the matching lines.

$ cat ip.txt
    * sky
    * apple
$ cat fav_color.txt
deep red
yellow
reddish
brown

# space between r and filename is optional
# add entire content of 'ip.txt' after each line containing 'red'
$ sed '/red/r ip.txt' fav_color.txt
deep red
    * sky
    * apple
yellow
reddish
    * sky
    * apple
brown

If the given filename doesn't exist, sed will silently ignore it and proceed as if the file was empty. Exit status will be 0 unless something else goes wrong with the sed command used.

To use a command output as the input file source, you can use /dev/stdin as the filename. However, once the content of /dev/stdin is exhausted, you won't get any more data. So, using /dev/stdin is not always functionally equivalent to providing a filename. As an example, compare the output for cat ip.txt | sed '/red/r /dev/stdin' fav_color.txt to sed '/red/r ip.txt' fav_color.txt. So, if you have multiple matches, save the command output to a file first and then use that filename with r instead of /dev/stdin.

$ text='good\tone\nfood\tpun'
$ echo "$text" | sed '1r /dev/stdin' ip.txt
    * sky
good\tone\nfood\tpun
    * apple

# example for adding multiline command output
$ seq 2 | sed '2r /dev/stdin' ip.txt
    * sky
    * apple
1
2

# note that newline won't be added to the file content being read
$ printf '123' | sed '1r /dev/stdin' ip.txt
    * sky
123    * apple

You can use 0 as the address to insert file contents before the first line. This feature was added in GNU sed 4.9 to the r command.

$ sed '0r ip.txt' greeting.txt
    * sky
    * apple
Hi there
Have a nice day

Here are some examples to emulate the change command functionality. As seen in the previous chapter, use the -e option or literal newlines when multiple commands are needed. See also unix.stackexchange: Various ways to replace line M in file1 with line N in file2.

# replace only the 2nd line
# order is important, first 'r' and then 'd'
# note the use of command grouping to avoid repeating the address
$ items='    * blue\n    * green\n'
$ printf '%b' "$items" | sed -e '2 {r /dev/stdin' -e 'd}' ip.txt
    * sky
    * blue
    * green

# replace range of lines
# command grouping will add file contents for each matching line
# so, use 'r' only for one of the addresses and then delete the range
# // here avoids address duplication
# same as: sed -e '/^red/r ip.txt' -e '/yellow/,//d' fav_color.txt
$ sed -e '/yellow/r ip.txt' -e '//,/^red/d' fav_color.txt
deep red
    * sky
    * apple
brown

Quoting from the manual:

The empty regular expression // repeats the last regular expression match (the same holds if the empty regular expression is passed to the s command). Note that modifiers to regular expressions are evaluated when the regular expression is compiled, thus it is invalid to specify them together with the empty regular expression

Emulating insert command functionality with the r command requires advanced usage of sed, which is beyond the scope of this book. See unix.stackexchange: insert file contents before matching line for examples. Instead, the next section will show how to use the e flag for this purpose.

Insert file using e and cat

The manual has this handy note for the e flag:

Note that, unlike the r command, the output of the command will be printed immediately; the r command instead delays the output to the end of the current cycle.

This makes the e flag the easiest way to insert file content before the matching lines. Similar to the r command, the output of an external command is inserted literally. But one difference from r command is that if the filename passed to the external cat command doesn't exist, then you will see its error message inserted.

$ sed '/red/e cat ip.txt' fav_color.txt
    * sky
    * apple
deep red
yellow
    * sky
    * apple
reddish
brown

Append line by line using R

The R command is very similar to the r command. But instead of reading the entire file content, R will read one line at a time from the source file when the address matches. If the entire file has already been consumed and another match is found, sed will proceed as if the next line to read was empty.

$ sed '/red/R ip.txt' fav_color.txt
deep red
    * sky
yellow
reddish
    * apple
brown

# interleave contents of two files
$ seq 4 | sed 'R /dev/stdin' fav_color.txt
deep red
1
yellow
2
reddish
3
brown
4

See also stackoverflow: Replace first few lines with first few lines from other file.

Cheatsheet and summary

This chapter showed robust solutions for adding text literally from a file or command output. These are particularly useful for templating solutions where a line containing a keyword gets replaced with text from elsewhere. In the next chapter, you'll learn how to implement control structures using branch commands.

Exercises

The exercises directory has all the files used in this section.

1) For the input file addr.txt, replace from the third to fifth lines with the second to fourth lines from para.txt.

$ sed ##### add your solution here
Hello World
How are you
Start working on that
project you always wanted
to, do not let it end
You are funny

2) Add one line from hex.txt after every two lines of copyright.txt.

$ sed ##### add your solution here
bla bla 2015 bla
blah 2018 blah
start address: 0xA0, func1 address: 0xA0
bla bla bla
copyright: 2019
end address: 0xFF, func2 address: 0xB0

3) For every line of the input file hex.txt, insert --- before the line and append one line from replace.txt as shown below.

$ sed ##### add your solution here
---
start address: 0xA0, func1 address: 0xA0
0xA0 0x5000
---
end address: 0xFF, func2 address: 0xB0
0xB0 0x6000

4) Insert the contents of hex.txt before a line matching 0x6000 of the input file replace.txt.

$ sed ##### add your solution here
0xA0 0x5000
start address: 0xA0, func1 address: 0xA0
end address: 0xFF, func2 address: 0xB0
0xB0 0x6000
0xFF 0x7000

5) For the input file addr.txt, replace lines containing are with contents of hex.txt.

$ sed ##### add your solution here
Hello World
start address: 0xA0, func1 address: 0xA0
end address: 0xFF, func2 address: 0xB0
This game is good
Today is sunny
12345
start address: 0xA0, func1 address: 0xA0
end address: 0xFF, func2 address: 0xB0

6) The two commands shown below are equivalent. True or False?

sed '/are/r pets.txt' addr.txt

cat pets.txt | sed '/are/r /dev/stdin' addr.txt

7) What do the commands sed '0r file1' file2 and sed '0R file1' file2 do?