/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote185 views1 answer
How to make a tuple
Python
0 vote172 views1 answer
How to define an empyt dic tin python
Python
2 votes169 views4 answers
Eval function
Python
1 vote185 views1 answer
[Solved] TypeError: can’t multiply sequence by non-int of type str
Python
9 votes186 views1 answer
Override python print for class
Python
0 vote200 views1 answer
Failed to build wxPython
Python
0 vote163 views1 answer
Np.conjugate
Python
0 vote172 views1 answer
Matrix diagonal sum leetcode in Python
Python
0 vote170 views1 answer
Array with zeros python
Python
4 votes179 views4 answers
Bar chart in python
Python