/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote206 views1 answer
Spotify recommendations
Python
2 votes189 views1 answer
How to add headers in csv file using python
Python
0 vote192 views1 answer
Fillna spark dataframe
Python
6 votes201 views3 answers
Change current working directory python
Python
2 votes228 views1 answer
How to redirect in flask to the same page
Python Flask
0 vote192 views2 answers
Fibonacci sequence algorithm python
Python
0 vote189 views1 answer
Create image tkinter set active background
Python
2 votes179 views1 answer
Copy website
Python
0 vote172 views1 answer
Ordenar lista decrescente python
Python
12 votes262 views6 answers
Python open pickle file
Python