Pandas parallelize for loop

import multiprocessing as mp
pool = mp.Pool(processes=mp.cpu_count())

def func( arg ):
    idx,row = arg
	# Edit df
    return row

new_rows = pool.map( func, [(idx,row) for idx,row in data_all.iterrows()])
data_all_new = pd.concat( new_rows )

Source: stackoverflow.com

Similar codes

1 vote132 views1 answer
Streamlit button to load a file
Python
0 vote126 views1 answer
Kinesis Client get_records response json
Python
71 votes123 views6 answers
Alphabet list python
Python
0 vote130 views4 answers
Client network socket disconnected before secure TLS connection was established
Python
0 vote139 views1 answer
How to draw a square pygame
Python
52 votes116 views3 answers
How to make a venv python
Python
0 vote160 views1 answer
Django pointfield value format for fixtures
Python Django
0 vote125 views1 answer
Scrapy get raw html content of selector innerhtml
Python
-1 vote125 views1 answer
How do you split a list into evenly sized chunks?
Python
0 vote130 views1 answer
Use a callable instead, e.g., use `dict` instead of `{}`
Python