/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote148 views1 answer
Make a bot that spams space bar python
Python
0 vote180 views1 answer
Python read file to eof
Python
7 votes160 views1 answer
Python Emoji
Python
4 votes153 views2 answers
Finding median on python
Python
0 vote142 views1 answer
Python datetime subtract seconds
Python
2 votes161 views1 answer
Add a list in python
Python
0 vote156 views1 answer
Time counter in python
Python
0 vote168 views1 answer
Pyglet on close
Python
0 vote152 views1 answer
Initiate matrix python with zeros
Python
0 vote124 views1 answer
Multiplication of matrix in python
Python