/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

6 votes173 views1 answer
Button in flask
Python Flask
1 vote120 views3 answers
Python find string count in str
Python
0 vote112 views1 answer
Python program to assess if a file is closed or not
Python
0 vote119 views1 answer
"status": 400, "message": "The string was not recognized as a valid DateTime
Python
3 votes111 views1 answer
Width and height of pil image
Python
0 vote116 views1 answer
Pandas print tabulate no index
Python
1 vote120 views1 answer
Drop a row with a specific value of a column
Python
0 vote99 views1 answer
Python slice a list a specific amount of times
Python
0 vote122 views1 answer
How to fix snap "pycharm-community" has "install-snap" change in progress
Python
0 vote135 views1 answer
Boolean indexing datetime object | converting string to datetime object
Python