/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

3 votes141 views1 answer
How to encrypt a string python
Python
-1 vote151 views2 answers
Install unity on pop os
Python
3 votes166 views2 answers
Flask db migrate
Python
0 vote186 views1 answer
Removing a channel from aconda
Python
0 vote148 views1 answer
Value Error handling
Python
2 votes147 views1 answer
Kivy changing screen in python
Python
0 vote162 views1 answer
Pyglet on close
Python
0 vote147 views1 answer
Access host database django docker
Python
4 votes142 views2 answers
Change django admin title
Python
4 votes166 views1 answer
How to import reverse
Python