Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

118 votes145 views6 answers
Python download
Python
0 vote145 views1 answer
Python read from stdin pipe
Python
0 vote141 views1 answer
Actual python iterators
Python
0 vote159 views1 answer
Numpy array majority and how many
Python
1 vote163 views1 answer
Esp8266 micropython ds18b20
Python
1 vote209 views1 answer
How to make an simple python client
Python
27 votes148 views2 answers
Auto-py-to-exe with python3
Python
15 votes134 views3 answers
Not in pandas condition
Python
8 votes135 views1 answer
How to find no of times a elements in list python
Python
0 vote161 views2 answers
Draw circles matplotlib
Python