/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote105 views3 answers
Join list with comma python
Python
3 votes114 views1 answer
Count number items in list python
Python
0 vote132 views2 answers
Remove element from a list python
Python
0 vote124 views1 answer
ImportError: No module named _bootlocale
Python
0 vote127 views1 answer
Clahe opencv
Python
0 vote111 views1 answer
Long type python
Python
0 vote111 views1 answer
Numpy index array all except
Python
0 vote107 views2 answers
How to return a missing element in python
Python
0 vote125 views1 answer
Python read excel sheet name
Python
0 vote111 views1 answer
Convert list to tree python
Python