Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote115 views1 answer
Python first two numbers
Python
0 vote126 views1 answer
Pygame image get height
Python
0 vote140 views1 answer
Geopandas dataframe to ogr layer
Python
0 vote126 views1 answer
Python TypeError: function takes positional arguments but were given
Python
14 votes107 views2 answers
What does related name do in django
Python
20 votes121 views4 answers
Add input to list python
Python
7 votes166 views1 answer
Np array n same values
Python
14 votes154 views5 answers
Permutations python
Python
0 vote129 views1 answer
Distance between numpy arrays
Python
0 vote158 views1 answer
Python argparse only allow certain values
Python