Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

1 vote159 views1 answer
Git clone in python to tmp directory
Python
0 vote138 views1 answer
Pandas save csv list as columns
Python
0 vote104 views1 answer
\r\n python
Python
5 votes139 views2 answers
How to say that an input needs to be a number python
Python
0 vote113 views1 answer
Find and replace subword in word python regex
Python
44 votes107 views3 answers
Python convert string to bytes
Python
0 vote127 views1 answer
Pyqt loading screen
Python
4 votes111 views4 answers
Download pdf python
Python
4 votes113 views3 answers
Python create new pandas dataframe with specific columns
Python
0 vote112 views1 answer
How to apply 1nf dataframe in python
Python