Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote151 views1 answer
Remove ï»¿ python
Python
0 vote150 views4 answers
Roobet crash bot
Python
0 vote149 views2 answers
How to add space in python
Python
1 vote154 views1 answer
Subprocess print logs
Python
3 votes147 views2 answers
Editing specific line in text file in python
Python
0 vote152 views1 answer
Btr
Python
0 vote145 views1 answer
How to get the top 100 frequent words on a python dataframe colummn
Python
0 vote185 views1 answer
List to string without loop
Python
0 vote108 views1 answer
Pygame triangle
Python
0 vote141 views1 answer
Pandas nat to null?
Python