Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

41 votes120 views6 answers
Pd.read_csv
Python
1 vote117 views1 answer
Python web parser
Python
0 vote116 views1 answer
Activate inherit function django
Python
1 vote123 views1 answer
Write csv python pandas stack overflow
Python
13 votes131 views5 answers
Get index of item in list
Python
2 votes104 views2 answers
Django admin actions
Python Django
1 vote119 views1 answer
Get a column that corresponds to the average of two columns pandas
Python
0 vote119 views1 answer
Group by in ruby mongoid
Python
-2 votes124 views2 answers
Get sum of 2d array python
Python
0 vote138 views2 answers
Get ContentType with django get_model
Python