Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote217 views1 answer
Django Signup urls.py
Python
0 vote186 views1 answer
Install os conda
Python
0 vote179 views1 answer
How to read comment before the root element of xml python
Python
0 vote164 views1 answer
ImportError: cannot import name 'MigrateCommand' from 'flask_migrate'
Python
2 votes164 views1 answer
Calculate market value crsp pandas
Python
0 vote166 views1 answer
Python offline translate pypi
Python
20 votes166 views2 answers
How to print all rows in pandas
Python
0 vote163 views1 answer
Pandas print dataframe dtypes
Python
11 votes205 views5 answers
Pandas dataframe delete column
Python
21 votes220 views2 answers
Run python file using python code
Python