Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

-2 votes188 views1 answer
Python sort list string
Python
1 vote190 views6 answers
Tuple python
Python
0 vote187 views2 answers
Examples of function decorators in Python
Python
1 vote166 views1 answer
Export python pandas dataframe as json file
Python
2 votes195 views1 answer
Termcolor python
Python
1 vote187 views1 answer
Python sum attribute in list
Python
0 vote152 views1 answer
How to query json field django
Python
22 votes168 views5 answers
Python palindrome check
Python
31 votes166 views3 answers
How to add one to the index of a list
Python
1 vote177 views1 answer
How to add value to to interger in python
Python