Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

1 vote261 views1 answer
Python program to find fibonacci series using function recursion loop
Python
24 votes193 views5 answers
Load static
Python
0 vote208 views2 answers
Convert uint8 to double in python
Python
0 vote203 views1 answer
Skills=request.POST['skills']
Python
0 vote194 views1 answer
What is xarray
Python
1 vote214 views1 answer
Img read
Python
20 votes193 views4 answers
Check django version windows
Python
17 votes200 views2 answers
Discord bot status python
Python
0 vote197 views3 answers
Python datetime add month
Python
0 vote198 views1 answer
Python : a counter
Python