Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

26 votes164 views2 answers
Pil image crop
Python
0 vote230 views2 answers
Django run manage.py from python
Python
1 vote241 views1 answer
Unresolved reference 'django'
Python
8 votes216 views3 answers
How to install whl file in python
Python
0 vote189 views1 answer
Semaphore in python
Python
13 votes211 views2 answers
Get start time and end time in python
Python
1 vote179 views1 answer
Difference between 2 timestamps pandas
Python
1 vote205 views1 answer
Plotly line plot
Python
1 vote289 views1 answer
How to custom page not found in django
Python Django
16 votes148 views1 answer
Save model keras
Python