Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

6 votes133 views5 answers
Mean python
Python
-1 vote182 views1 answer
Pandas splitting the data based on the day type
Python
0 vote136 views1 answer
How to get the date from week number in Python?
Python
1 vote175 views1 answer
Django datetimefield default
Python Django
1 vote129 views3 answers
Char list to string python
Python
0 vote135 views1 answer
How to check if user pressed enter in python
Python
0 vote133 views1 answer
Multiple assessment in python
Python
0 vote132 views1 answer
Open url from ipywidgets
Python
107 votes133 views6 answers
Python do while loop
Python
0 vote129 views1 answer
Compute the 2d histogram of x and y.
Python