/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

2 votes181 views2 answers
How to Add a Title to Seaborn Plots
Python
0 vote173 views1 answer
Global /tmp/pip-req-build-civioau0/opencv/modules/videoio/src/cap_v4l.cpp (587) autosetup_capture_mode_v4l2 VIDEOIO(V4L2:/dev/video0): device is busy
Python
0 vote187 views1 answer
No module named 'neat
Python
26 votes164 views6 answers
Delete columns pandas
Python
11 votes175 views5 answers
Dataframe change index
Python
80 votes204 views21 answers
Python syntax cheat sheet
Python
0 vote184 views1 answer
ValueError: Tried to convert 'shape' to a tensor and failed. Error: None values not supported.
Python
1 vote202 views1 answer
Usong brave browser pyhton
Python
0 vote212 views1 answer
If statement collection python
Python
3 votes271 views5 answers
Euclid algorithm
Python