/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

0 vote194 views2 answers
Grep second match
Shell/Bash
0 vote222 views1 answer
Crontab is not running my shell script
Shell/Bash
0 vote198 views1 answer
How to make a folder super user errno: -13, code: 'EACCES', syscall: 'lin ubuntu
Shell/Bash
1 vote281 views1 answer
Error cannot find module 'browser-sync' gulp
Shell/Bash
4 votes209 views1 answer
Verify parameter one bash
Shell/Bash
0 vote191 views1 answer
Right click touchpad ubuntu budgie
Shell/Bash
0 vote201 views1 answer
Grep get everything between two strings
Shell/Bash
1 vote207 views1 answer
How to send notification in linux
Shell/Bash
0 vote201 views1 answer
Vim code highlighting mac
Shell/Bash
0 vote212 views1 answer
How to uninstall a package installed using homebrew
Shell/Bash