/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote121 views1 answer
Check for strings as positive/negative - integer/float
Python
0 vote138 views1 answer
How to make a minute counter in python
Python
4 votes143 views1 answer
Find text between two strings regex python
Python
2 votes134 views2 answers
Flask 'export' is not recognized as an internal or external command, operable program or batch file.
Python
1 vote147 views1 answer
Python selenium web scraping example
Python
0 vote174 views1 answer
Extends template django file system
Python Django
0 vote134 views1 answer
Return more than one value python
Python
0 vote137 views3 answers
Python thread exception
Python
0 vote122 views1 answer
Data where values in column starts with particular value
Python
0 vote170 views1 answer
Seaborrn set figsize
Python