Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote201 views1 answer
Spacy Vietnamese
Python
5 votes186 views2 answers
Selenium scroll to element python
Python
0 vote204 views1 answer
Hello world with a variable in python 3
Python
3 votes201 views1 answer
Display flask across network
Python Flask
0 vote200 views1 answer
How to convert response to beautifulsoup object
Python
0 vote192 views1 answer
Fibonacci sequence in python using whileloop
Python
0 vote302 views2 answers
Generate-thumbnails-in-django-with-pil
Python Django
86 votes200 views6 answers
CSV.DictReader
Python
135 votes236 views6 answers
Replace string
Python
4 votes197 views2 answers
Python sort dictionary alphabetically by key
Python