This is a work-in-progress draft version.

huniq

From github: huniq:

Command line utility to remove duplicates from the given input. Note that huniq does not sort the input, it just removes duplicates.

Installation

See huniq: installation for details.

$ cargo install huniq

Removing duplicates

Sample input with duplicates:

$ cat purchases.txt
coffee
tea
washing powder
coffee
toothpaste
tea
soap
tea

The default command invocation of huniq removes duplicates. Input is not sorted, so input order is retained in the output.

$ huniq <purchases.txt
coffee
tea
washing powder
toothpaste
soap

Number of occurrences

Similar to uniq -c, you can use the -c option to get the number of occurrences for each entry. The output will be in random order.

$ huniq -c <purchases.txt
1 soap
1 washing powder
3 tea
1 toothpaste
2 coffee

Sort based on number of occurrences

Use -s option to sort the output based on number of occurrences in ascending order. Use -S for descending order. The -c option is implied for these two options.

$ huniq -s <purchases.txt
1 toothpaste
1 soap
1 washing powder
2 coffee
3 tea

$ huniq -S <purchases.txt
3 tea
2 coffee
1 toothpaste
1 washing powder
1 soap

Note that the order for entries with the same number of occurrences will be random.