/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

18 votes106 views6 answers
Linux commands
Shell/Bash
0 vote113 views1 answer
How to remove sudo permission in from download
Shell/Bash
1 vote100 views1 answer
Bash conditional sum
Shell/Bash
0 vote92 views1 answer
Heroku process
Shell/Bash
5 votes100 views2 answers
Git can we reset a new branch to a specific commit
Shell/Bash
3 votes101 views1 answer
Pm2 show command
Shell/Bash
1 vote74 views2 answers
Restart bluetooth ubuntu
Shell/Bash
0 vote118 views1 answer
Restart supervisor
Shell/Bash
0 vote114 views1 answer
Date last friedy in linux
Shell/Bash
6 votes59 views1 answer
Gitee change url
Shell/Bash