Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote235 views2 answers
Apply butterworth filter python
Python
0 vote218 views1 answer
K.set_image_dim_ordering('tf') AttributeError: module 'keras.backend' has no attribute 'set_image_dim_ordering'
Python
28 votes217 views2 answers
Python date add days
Python
0 vote201 views1 answer
Watershed Segmentation
Python
1 vote258 views3 answers
How to use variable from another function in python
Python
0 vote189 views1 answer
From decimal to binary python without using any libraries
Python
0 vote191 views1 answer
Python print list items vertically
Python
13 votes218 views1 answer
Python read file to variable
Python
0 vote268 views4 answers
Python convert dict_keys to list
Python
0 vote192 views1 answer
Bytearray to hex python
Python