/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

1 vote117 views1 answer
Set different user for git
Shell/Bash
15 votes121 views6 answers
Bash: nvm: command not found
Shell/Bash
14 votes147 views3 answers
Compress folder ubuntu
Shell/Bash
1 vote127 views1 answer
Shell script to reindex elasticsearch
Shell/Bash
1 vote119 views4 answers
Docker remove exited containers
Shell/Bash
1 vote110 views1 answer
Install ext-soap
Shell/Bash
0 vote111 views1 answer
Bash: syntax error near unexpected token 'do'
Shell/Bash
0 vote116 views1 answer
Github add all files/directories and subdirectories
Shell/Bash
8 votes113 views1 answer
Git log show summary
Shell/Bash
0 vote118 views1 answer
Can't push refs to remote. Try running 'Pull' first to integrate your changes.
Shell/Bash