Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote152 views1 answer
Python discord know message from bot
Python
0 vote164 views1 answer
Visitor IP address django
Python
0 vote134 views1 answer
Cv2 open blank window
Python
0 vote147 views1 answer
Python count one to ten
Python
0 vote155 views1 answer
Python session set cookies
Python
31 votes151 views3 answers
How to add one to the index of a list
Python
0 vote152 views1 answer
Tkinter lottery app
Python
0 vote137 views2 answers
How to record youtube cc in python
Python
2 votes202 views3 answers
Change python version of a conda environment
Python
4 votes142 views1 answer
Pandas dataframe replace inf
Python