Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote134 views1 answer
HOW TO CREATE A DATETIME LIST QUICK
Python
0 vote131 views1 answer
Currelation matrix python
Python
12 votes132 views6 answers
Add elements to list python
Python
102 votes138 views6 answers
Python contains
Python
0 vote142 views1 answer
Coercion python
Python
1 vote132 views1 answer
Django m2m .add
Python
2 votes143 views2 answers
Sort by multiple keys in object python
Python
0 vote150 views1 answer
Trim all new rows string python
Python
-1 vote97 views2 answers
Beautifulsoup docs
Python
0 vote134 views5 answers
Append in dictionary with matrix values
Python