cat and tac

cat derives its name from concatenation and provides other nifty options too.

tac helps you to reverse the input line wise, usually used for further text processing.

Creating text files

Yeah, cat can be used to write contents to a file by typing them from the terminal itself. If you invoke cat without providing file arguments or stdin data from a pipe, it will wait for you to type the content. After you are done typing all the text you want to save, press Enter and then the Ctrl+d key combinations. If you don't want the last line to have a newline character, press Ctrl+d twice instead of Enter and Ctrl+d. See also unix.stackexchange: difference between Ctrl+c and Ctrl+d.

# press Enter and Ctrl+d after typing all the required characters
$ cat > greeting.txt
Hi there
Have a nice day

In the above example, the output of cat is redirected to a file named greeting.txt. If you don't redirect the stdout data, each line will be echoed as you type. You can check the contents of the file you just created by using cat again.

$ cat greeting.txt
Hi there
Have a nice day

Here Documents is another popular way to create such files. In this case, the termination condition is a line matching a predefined string which is specified after the << redirection operator. This is especially helpful for automation, since pressing Ctrl+d interactively isn't desirable. Here's an example:

# > and a space at the start of lines represents the secondary prompt PS2
# don't type them in a shell script
# EOF is typically used as the identifier
$ cat << 'EOF' > fruits.txt
> banana
> papaya
> mango
> EOF

$ cat fruits.txt
banana
papaya
mango

The termination string is enclosed in single quotes to prevent parameter expansion, command substitution, etc. You can also use \string for this purpose. If you use <<- instead of <<, you can use leading tab characters for indentation purposes. See bash manual: Here Documents and stackoverflow: here-documents for more examples and details.

info Note that creating files as shown above isn't restricted to cat, it can be applied to any command waiting for stdin.

# 'tr' converts lowercase alphabets to uppercase in this example
$ tr 'a-z' 'A-Z' << 'end' > op.txt
> hi there
> have a nice day
> end

$ cat op.txt
HI THERE
HAVE A NICE DAY

Concatenate files

Here are some examples to showcase cat's main utility. One or more files can be passed as arguments.

$ cat greeting.txt fruits.txt nums.txt
Hi there
Have a nice day
banana
papaya
mango
3.14
42
1000

info Visit the cli_text_processing_coreutils repo to get all the example files used in this book.

To save the output of concatenation, use the shell's redirection features.

$ cat greeting.txt fruits.txt nums.txt > op.txt

$ cat op.txt
Hi there
Have a nice day
banana
papaya
mango
3.14
42
1000

Accepting stdin data

You can represent the stdin data using - as a file argument. If the file arguments are not present, cat will read the stdin data if present or wait for interactive input as seen earlier.

# only stdin (- is optional in this case)
$ echo 'apple banana cherry' | cat
apple banana cherry

# both stdin and file arguments
$ echo 'apple banana cherry' | cat greeting.txt -
Hi there
Have a nice day
apple banana cherry

# here's an example without a newline character at the end of the first input
$ printf 'Some\nNumbers' | cat - nums.txt
Some
Numbers3.14
42
1000

Squeeze consecutive empty lines

As mentioned before, cat provides many features beyond concatenation. Consider this sample stdin data:

$ printf 'hello\n\n\nworld\n\nhave a nice day\n\n\n\n\n\napple\n'
hello


world

have a nice day





apple

You can use the -s option to squeeze consecutive empty lines to a single empty line. If present, leading and trailing empty lines will also be squeezed (won't be completely removed). You can modify the below example to test it out.

$ printf 'hello\n\n\nworld\n\nhave a nice day\n\n\n\n\n\napple\n' | cat -s
hello

world

have a nice day

apple

Prefix line numbers

The -n option will prefix line numbers and a tab character to each input line. The line numbers are right justified to occupy a minimum of 6 characters, with space as the filler.

$ cat -n greeting.txt fruits.txt nums.txt
     1  Hi there
     2  Have a nice day
     3  banana
     4  papaya
     5  mango
     6  3.14
     7  42
     8  1000

Use the -b option instead of -n if you don't want empty lines to be numbered.

# -n option numbers all the input lines
$ printf 'apple\n\nbanana\n\ncherry\n' | cat -n
     1  apple
     2  
     3  banana
     4  
     5  cherry

# -b option numbers only the non-empty lines
$ printf 'apple\n\nbanana\n\ncherry\n' | cat -b
     1  apple

     2  banana

     3  cherry

info Use the nl command if you want more customization options like number formatting, separator string, regular expression based filtering and so on.

Viewing special characters

Characters like backspace and carriage return will mangle the contents if viewed naively on the terminal. Characters like NUL won't even be visible. You can use the -v option to show such characters using the caret notation (see wikipedia: Control code chart for details). See this unix.stackexchange thread for non-ASCII examples.

# example for backspace and carriage return characters
$ printf 'mar\bt\nbike\rp\n'
mat
pike
$ printf 'mar\bt\nbike\rp\n' | cat -v
mar^Ht
bike^Mp

# NUL character
$ printf 'car\0jeep\0bus\0' | cat -v
car^@jeep^@bus^@

# form-feed and vertical-tab
$ printf '1 2\t3\f4\v5\n' | cat -v
1 2     3^L4^K5

The -v option doesn't cover the newline and tab characters. You can use the -T option to spot tab characters.

$ printf 'good food\tnice dice\napple\tbanana\tcherry\n' | cat -T
good food^Inice dice
apple^Ibanana^Icherry

The -E option adds a $ marker at the end of input lines. This is useful to spot trailing whitespace characters.

$ printf 'ice   \nwater\n cool  \n chill\n' | cat -E
ice   $
water$
 cool  $
 chill$

The following options combine two or more of the above options:

  • -e option is equivalent to -vE
  • -t option is equivalent to -vT
  • -A option is equivalent to -vET
$ printf 'mar\bt\nbike\rp\n' | cat -e
mar^Ht$
bike^Mp$

$ printf '1 2\t3\f4\v5\n' | cat -t
1 2^I3^L4^K5

$ printf '1 2\t3\f4\v5\n' | cat -A
1 2^I3^L4^K5$

Useless use of cat

Using cat to view the contents of a file, to concatenate them, etc are well and good. But, using cat when it is not needed is a bad habit that you should avoid. See wikipedia: UUOC and Useless Use of Cat Award for more details.

Most commands that you'll see in this book can directly work with file arguments, so you shouldn't use cat to pipe the contents for such cases. Here's a single file example:

# useless use of cat
$ cat greeting.txt | sed -E 's/\w+/\L\u&/g'
Hi There
Have A Nice Day

# sed can handle file arguments
$ sed -E 's/\w+/\L\u&/g' greeting.txt
Hi There
Have A Nice Day

If you prefer having the file argument before the command, you can use the shell's redirection feature to supply input data instead of cat. This also applies to commands like tr that do not accept file arguments.

# useless use of cat
$ cat greeting.txt | tr 'a-z' 'A-Z'
HI THERE
HAVE A NICE DAY

# use shell redirection instead
$ <greeting.txt tr 'a-z' 'A-Z'
HI THERE
HAVE A NICE DAY

Such useless use of cat might not have a noticeable negative impact for most cases. But it becomes important if you are dealing with large input files. Especially for commands like tac and tail which will have to wait for all the data to be read instead of directly processing from the end of the file if they had been passed as arguments (or using shell redirection).

If you are dealing with multiple files, then the use of cat will depend upon the desired result. Here are some examples:

# match lines containing 'o' or '0'
# -n option adds line number prefix
$ cat greeting.txt fruits.txt nums.txt | grep -n '[o0]'
5:mango
8:1000
$ grep -n '[o0]' greeting.txt fruits.txt nums.txt
fruits.txt:3:mango
nums.txt:3:1000

# count the number of lines containing 'o' or '0'
$ grep -c '[o0]' greeting.txt fruits.txt nums.txt
greeting.txt:0
fruits.txt:1
nums.txt:1
$ cat greeting.txt fruits.txt nums.txt | grep -c '[o0]'
2

For some use cases like in-place editing with sed, you can't use cat or shell redirection at all. The files have to be passed as arguments only. To conclude, don't use cat just to pass the input as stdin to another command, unless necessary.

tac

tac will reverse the order of the input lines. If you pass multiple input files, each file content will be reversed separately. Here are some examples:

# won't be the same as: cat greeting.txt fruits.txt | tac
$ tac greeting.txt fruits.txt
Have a nice day
Hi there
mango
papaya
banana

$ printf 'apple\nbanana\ncherry\n' | tac
cherry
banana
apple

warning If the last input line doesn't end with a newline, the output will also not have that newline character.

$ printf 'apple\nbanana\ncherry' | tac
cherrybanana
apple

Reversing input lines makes some of the text processing tasks easier. For example, if there are multiple matches but you want only the last one. See my ebooks on GNU sed and GNU awk for more such use cases.

$ cat log.txt
--> warning 1
a,b,c,d
42
--> warning 2
x,y,z
--> warning 3
4,3,1

$ tac log.txt | grep -m1 'warning'
--> warning 3

$ tac log.txt | sed '/warning/q' | tac
--> warning 3
4,3,1

In the above example, log.txt has multiple lines containing warning. The task is to fetch lines based on the last match, which isn't usually supported by CLI tools. Matching the first occurrence is easy with tools like grep and sed. Hence, tac is helpful to reverse the condition from the last match to the first match. After processing with tools like sed, the result is then reversed again to get back the original order of input lines. Another benefit is that the first tac command will stop reading the input contents after the match is found.

info Use the rev command if you want each input line to be reversed character wise.

Customize line separator for tac

By default, the newline character is used to split the input content into lines. You can use the -s option to specify a different string to be used as the separator.

# use NUL as the line separator
# -s $'\0' can also be used instead of -s '' if ANSI-C quoting is supported
$ printf 'car\0jeep\0bus\0' | tac -s '' | cat -v
bus^@jeep^@car^@

# as seen before, the last entry should also have the separator
# otherwise it won't be present in the output
$ printf 'apple banana cherry' | tac -s ' ' | cat -e
cherrybanana apple $
$ printf 'apple banana cherry ' | tac -s ' ' | cat -e
cherry banana apple $

When the custom separator occurs before the content of interest, use the -b option to print those separators before the content in the output as well.

$ cat body_sep.txt
%=%=
apple
banana
%=%=
teal
green

$ tac -b -s '%=%=' body_sep.txt
%=%=
teal
green
%=%=
apple
banana

The separator will be treated as a regular expression if you use the -r option as well.

$ cat shopping.txt
apple   50
toys    5
Pizza   2
mango   25
Banana  10

# separator character is 'a' or 'm' at the start of a line
$ tac -b -rs '^[am]' shopping.txt
mango   25
Banana  10
apple   50
toys    5
Pizza   2

# alternate solution for: tac log.txt | sed '/warning/q' | tac
# separator is zero or more characters from the start of a line till 'warning'
$ tac -b -rs '^.*warning' log.txt | awk '/warning/ && ++c==2{exit} 1'
--> warning 3
4,3,1

info See Regular Expressions chapter from my GNU grep ebook if you want to learn about regexp syntax and features.

Exercises

info All the exercises are also collated together in one place at Exercises.md. For solutions, see Exercise_solutions.md.

info The exercises directory has all the files used in this section.

1) The given sample data has empty lines at the start and end of the input. Also, there are multiple empty lines between the paragraphs. How would you get the output shown below?

# note that there's an empty line at the end of the output
$ printf '\n\n\ndragon\n\n\n\nunicorn\nbee\n\n\n' | ##### add your solution here

     1  dragon

     2  unicorn
     3  bee

2) Pass appropriate arguments to the cat command to get the output shown below.

$ cat greeting.txt
Hi there
Have a nice day

$ echo '42 apples and 100 bananas' | cat ##### add your solution here
42 apples and 100 bananas
Hi there
Have a nice day

3) What does the -v option of the cat command do?

4) Which options of the cat command do the following stand in for?

  • -e option is equivalent to
  • -t option is equivalent to
  • -A option is equivalent to

5) Will the two commands shown below produce the same output? If not, why not?

$ cat fruits.txt ip.txt | tac

$ tac fruits.txt ip.txt

6) Reverse the contents of blocks.txt file as shown below, considering ---- as the separator.

$ cat blocks.txt
----
apple--banana
mango---fig
----
3.14
-42
1000
----
sky blue
dark green
----
hi hello

##### add your solution here
----
hi hello
----
sky blue
dark green
----
3.14
-42
1000
----
apple--banana
mango---fig

7) For the blocks.txt file, write solutions to display only the last such group and last two groups.

##### add your solution here
----
hi hello

##### add your solution here
----
sky blue
dark green
----
hi hello

8) Reverse the contents of items.txt as shown below. Consider digits at the start of lines as the separator.

$ cat items.txt
1) fruits
apple 5
banana 10
2) colors
green
sky blue
3) magical beasts
dragon 3
unicorn 42

##### add your solution here
3) magical beasts
dragon 3
unicorn 42
2) colors
green
sky blue
1) fruits
apple 5
banana 10