/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

0 vote145 views1 answer
Linux find newer recent files reference to certain date
Shell/Bash
0 vote122 views1 answer
C pipe 2 arguments
Shell/Bash
0 vote142 views1 answer
Git delete master branch and recreate
Shell/Bash
1 vote142 views1 answer
Wget download list of urls
Shell/Bash
0 vote129 views1 answer
Execution failed for task ':react-native-firebase_auth:generateDebugRFile'
Shell/Bash
16 votes138 views6 answers
Ubuntu install tar.xz
Shell/Bash
0 vote377 views1 answer
Mpeg-4 aac decoder plugin for linux download
Shell/Bash
1 vote138 views1 answer
Install python3 raspberry pi
Shell/Bash
2 votes146 views1 answer
Kubectl pod stuck terminating
Shell/Bash
0 vote145 views1 answer
Git stash apply previous
Shell/Bash