/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

-1 vote118 views1 answer
How to convert pandas series to 2d numpy array
Python
0 vote129 views1 answer
Input and ouput array in python
Python
0 vote123 views1 answer
Can list hold different data types in python
Python
6 votes159 views4 answers
Python Day of the week
Python
3 votes126 views5 answers
Euclid's algorithm
Python
0 vote128 views3 answers
Python convert dat file to csv
Python
0 vote119 views1 answer
Django month name from month number
Python
0 vote127 views1 answer
Python how to add picture to label with tkinter
Python
0 vote129 views1 answer
Python string: escaping characters
Python
0 vote134 views1 answer
Python transform two columns to a list combine
Python