/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

11 votes150 views2 answers
Set os environment variable python
Python
0 vote137 views1 answer
Python Script to check how many images are broken
Python
0 vote164 views1 answer
Cv2 leave only the biggest blob
Python
1 vote135 views1 answer
Ros beginner tutorial python publisher
Python
0 vote158 views1 answer
Python String to array using list() method
Python
0 vote119 views0 answer
Python
2 votes142 views2 answers
Scipy cosine similarity
Python
-1 vote139 views1 answer
How to remove an element from dictionary using his value python
Python
-1 vote150 views2 answers
Python diagonal sum
Python
0 vote146 views1 answer
Insert-cells-in-empty-pandas-dataframe
Python