Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote138 views1 answer
Default style matplotlib python
Python
6 votes156 views3 answers
Python how to draw a square
Python
15 votes156 views3 answers
Count number of occurrences of all elements in list python
Python
0 vote151 views1 answer
How to auto create a three dimensional array in python
Python
-2 votes154 views6 answers
Django download
Python
0 vote128 views1 answer
Create a DataFrame containing elements in a range
Python
1 vote172 views1 answer
Find index of max value in 2d array python
Python
0 vote144 views1 answer
How to check if digit in int python
Python
0 vote133 views1 answer
2 plater die game in python
Python
2 votes144 views1 answer
Matplotlib plot remove margins
Python