/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote83 views6 answers
Assign dictionary a value from a list python
Python
6 votes112 views6 answers
Pandas if python
Python
0 vote109 views1 answer
Functools.cached_property objects in python
Python
3 votes108 views2 answers
Python list copy
Python
1 vote111 views1 answer
What does [::-1] mean in python
Python
3 votes103 views3 answers
NameError: name ‘pd’ is not defined
Python
0 vote235 views1 answer
Change colorbar size and place python
Python
1 vote97 views2 answers
Unknown amount of arguments discord py
Python
0 vote118 views1 answer
Inpunt and output
Python
0 vote218 views1 answer
Telethon get all channels
Python