/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote150 views1 answer
Get all subsets of a list python
Python
0 vote115 views3 answers
Float
Python
21 votes150 views6 answers
How to disable right click in js
Python
0 vote117 views1 answer
Django x-frame-options allowall
Python
1 vote111 views1 answer
Panda dataframe read csv change string to float
Python
0 vote128 views1 answer
How to add static files in django 3.2.9
Python
9 votes115 views4 answers
Display full dataframe pandas
Python
0 vote129 views1 answer
Ubuntu install pip for python 3.8
Python
0 vote143 views1 answer
Django less than and greater than
Python Django
1 vote126 views1 answer
Id() python
Python