Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

1 vote139 views1 answer
Turn of axis
Python
0 vote115 views1 answer
Django get latest object
Python
0 vote170 views1 answer
Backend not found Request Method: GET Request URL: http://127.0.0.1:8000/oauth/login/google-oauth2/ Raised by: social_django.views.au
Python Django
0 vote157 views4 answers
Read file from google drive colab
Python
12 votes143 views3 answers
Screen size python
Python
0 vote145 views1 answer
Math domain error python
Python
0 vote273 views1 answer
Python args description multiple lines
Python
1 vote118 views2 answers
Python map list of int to string
Python
0 vote197 views1 answer
2--2 in python prints?
Python
1 vote145 views1 answer
Xarray add coordinate
Python