/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

0 vote134 views1 answer
Bbedit close without saving
Shell/Bash
0 vote122 views1 answer
Fix mp3 file
Shell/Bash
0 vote205 views1 answer
Bash add extension to all files
Shell/Bash
0 vote121 views1 answer
Bash ls show link
Shell/Bash
2 votes116 views1 answer
Grep before and after
Shell/Bash
-1 vote118 views2 answers
Lsb release command centos
Shell/Bash
0 vote125 views1 answer
Generate patch file git
Shell/Bash
8 votes117 views5 answers
How to install node_module
Shell/Bash
0 vote157 views1 answer
Fast rename fasta header
Shell/Bash
0 vote116 views1 answer
Apt-get install ps
Shell/Bash