awk is handy to compare records and fields between two or more files. The key features used in the solution below:

  • For two files as input, NR==FNR will be true only when the first file is being processed
  • next will skip rest of the script and fetch the next record
  • a[$0] by itself is a valid statement. It will create an uninitialized element in array a with $0 as the key (assuming the key doesn't exist yet)
  • $0 in a checks if the given string ($0 here) exists as a key in the array a
$ cat colors_1.txt
teal
light blue
green
yellow
$ cat colors_2.txt
light blue
black
dark green
yellow

# common lines
$ awk 'NR==FNR{a[$0]; next} $0 in a' colors_1.txt colors_2.txt
light blue
yellow

# lines from colors_2.txt not present in colors_1.txt
$ awk 'NR==FNR{a[$0]; next} !($0 in a)' colors_1.txt colors_2.txt
black
dark green

warning Note that the NR==FNR logic will fail if the first file is empty, since NR wouldn't get a chance to increment. You can set a flag after the first file has been processed to avoid this issue. See this unix.stackexchange thread for more workarounds.

# no output
$ awk 'NR==FNR{a[$0]; next} !($0 in a)' /dev/null <(seq 2)

# gives the expected output
$ awk '!f{a[$0]; next} !($0 in a)' /dev/null f=1 <(seq 2)
1
2

Here's an example of comparing specific fields instead of whole lines. When you use a , separator between strings to construct the array key, the value of SUBSEP is inserted. This special variable has a default value of the non-printing character \034 which is usually not used as part of text files.

$ cat marks.txt
Dept    Name    Marks
ECE     Raj     53
ECE     Joel    72
EEE     Moi     68
CSE     Surya   81
EEE     Tia     59
ECE     Om      92
CSE     Amy     67

$ cat dept_name.txt
EEE Moi
CSE Amy
ECE Raj

$ awk 'NR==FNR{a[$1,$2]; next} ($1,$2) in a' dept_name.txt marks.txt
ECE     Raj     53
EEE     Moi     68
CSE     Amy     67

Video demo:


info See also my CLI text processing with GNU awk ebook.