/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

0 vote149 views1 answer
Open file in note from command line linux
Shell/Bash
1 vote149 views1 answer
Bash return unique lines starting at nth field
Shell/Bash
0 vote131 views1 answer
How to install atom on manjaro
Shell/Bash
2 votes136 views1 answer
Bash generate random number between
Shell/Bash
59 votes152 views2 answers
Git config user settup
Shell/Bash
3 votes155 views2 answers
Search file in ubuntu
Shell/Bash
0 vote155 views1 answer
Vnc not working ubuntu dconf
Shell/Bash
0 vote149 views1 answer
Install puppeteersharp
Shell/Bash
0 vote134 views1 answer
Libssl-dev install on ubuntu
Shell/Bash
0 vote166 views1 answer
Bash check return of command not error
Shell/Bash