paste
paste
is typically used to merge two or more files column wise. It also has a handy feature for serializing data.
Concatenating files column wise
Consider these two input files:
$ cat colors_1.txt
Blue
Brown
Orange
Purple
Red
Teal
White
$ cat colors_2.txt
Black
Blue
Green
Orange
Pink
Red
White
By default, paste
adds a tab character between corresponding lines of input files.
$ paste colors_1.txt colors_2.txt
Blue Black
Brown Blue
Orange Green
Purple Orange
Red Pink
Teal Red
White White
You can use the -d
option to change the delimiter between the columns. The separator is added even if the data has been exhausted for some of the input files. Here's some examples with single character delimiters, multicharacter separation will be discussed later.
$ seq 5 | paste -d, - <(seq 6 10)
1,6
2,7
3,8
4,9
5,10
# quote the delimiter if it is a shell metacharacter
$ paste -d'|' <(seq 3) <(seq 4 5) <(seq 6 8)
1|4|6
2|5|7
3||8
Use empty string if you don't want any delimiter between the columns. You can also use \0
for this case, but that'd be confusing since it is typically used to mean the NUL character.
# note that the space between -d and empty string is necessary here
$ paste -d '' <(seq 3) <(seq 6 8)
16
27
38
You can pass the same filename multiple times too, they will be treated as if they are separate inputs. This doesn't apply for
stdin
though, which is a special case as discussed in a later section.
Interleaving lines
By setting the newline character as the delimiter, you'll get interleaved lines.
$ paste -d'\n' <(seq 11 13) <(seq 101 103)
11
101
12
102
13
103
Multiple columns from single input
If you use -
multiple times, paste
will consume a line from stdin
data every time -
is encountered. This is different from using the same filename multiple times, in which case they are treated as separate inputs.
This special case for stdin
data is useful to combine consecutive lines using the given delimiter. Here's some examples to help you understand this feature better:
# two columns
$ seq 10 | paste -d, - -
1,2
3,4
5,6
7,8
9,10
# five columns
$ seq 10 | paste -d: - - - - -
1:2:3:4:5
6:7:8:9:10
# use input redirection for file input
$ <colors_1.txt paste -d: - - -
Blue:Brown:Orange
Purple:Red:Teal
White::
Here's an example with both stdin
and file arguments:
$ seq 6 | paste - nums.txt -
1 3.14 2
3 42 4
5 1000 6
If you don't want to manually type the number of -
required, you can use this printf
trick:
# the string before %.s is repeated based on the number of arguments
$ printf 'x %.s' a b c
x x x
$ printf -- '- %.s' {1..5}
- - - - -
$ seq 10 | paste -d, $(printf -- '- %.s' {1..5})
1,2,3,4,5
6,7,8,9,10
See this stackoverflow thread for more details about the
printf
solution and other alternatives.
Multicharacter delimiters
The -d
option accepts a list of characters (bytes to be precise) to be used one by one between the different columns. If the number of characters is less than the number of separators required, the characters are reused from the beginning and this cycle repeats until all the columns are done. If the number of characters is greater than the number of separators required, the extra characters are simply discarded.
# , is used between 1st and 2nd column
# - is used between 2nd and 3rd column
$ paste -d',-' <(seq 3) <(seq 4 6) <(seq 7 9)
1,4-7
2,5-8
3,6-9
# only 3 separators are needed, the rest are discarded
$ paste -d',-:;.[]' <(seq 3) <(seq 4 6) <(seq 7 9) <(seq 10 12)
1,4-7:10
2,5-8:11
3,6-9:12
# 2 characters given, 4 separators needed
# paste will reuse from the start of the list
$ seq 10 | paste -d':,' - - - - -
1:2,3:4,5
6:7,8:9,10
You can use empty files to get multicharacter separation between the columns.
$ paste -d' : ' <(seq 3) /dev/null /dev/null <(seq 4 6)
1 : 4
2 : 5
3 : 6
# create empty file to avoid typing /dev/null too many times
$ > e
$ paste -d' : - ' <(seq 3) e e <(seq 4 6) e e <(seq 7 9)
1 : 4 - 7
2 : 5 - 8
3 : 6 - 9
Serialize
The -s
option allows you to combine all the input lines from a file into a single line using the given delimiter. paste
will ensure to add a final newline character even if it isn't present in the input.
# <colors_1.txt tr '\n' ',' will give you a trailing comma
# paste changes the separator between the lines only
$ paste -sd, colors_1.txt
Blue,Brown,Orange,Purple,Red,Teal,White
# newline gets added at the end even if not present in the input
$ printf 'apple\nbanana\ncherry' | paste -sd-
apple-banana-cherry
If multiple files are passed, serialization of each file is displayed on separate lines.
$ paste -sd: colors_1.txt colors_2.txt
Blue:Brown:Orange:Purple:Red:Teal:White
Black:Blue:Green:Orange:Pink:Red:White
$ paste -sd, <(seq 3) <(seq 5 9)
1,2,3
5,6,7,8,9
NUL separator
Use -z
option if you want to use NUL character as the line separator. In this scenario, paste
will ensure to add a final NUL character even if not present in the input.
$ printf 'a\0b\0c\0d\0' | paste -z -d: - - | cat -v
a:b^@c:d^@