/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

12 votes165 views2 answers
Install pycharm ubuntu 20.04
Shell/Bash
0 vote191 views1 answer
How to check whether git is initialized or not
Shell/Bash
1 vote204 views1 answer
Set default branch to main on git init
Shell/Bash
2 votes180 views1 answer
Mamp connect to mysql
Shell/Bash
34 votes224 views6 answers
Sublime text linux
Shell/Bash
2 votes207 views1 answer
Redirect stderr to stdout
Shell/Bash
37 votes230 views2 answers
Ubuntu pip3
Shell/Bash
0 vote182 views1 answer
Hibernate command in windows 10 stackoverflow
Shell/Bash
0 vote187 views1 answer
Npm audit only high vuln
Shell/Bash
0 vote186 views1 answer
ModuleNotFoundError: No module named 'slack'
Shell/Bash