Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote134 views1 answer
Write a python program to find the most frequent word in text file
Python
83 votes96 views6 answers
Py virtual environment
Python
0 vote146 views1 answer
Python hewwo world
Python
2 votes175 views3 answers
Run flask in debug mode
Python Flask
5 votes160 views3 answers
How to hide command console python
Python
1 vote190 views1 answer
Python pickle save and load multiple variables
Python
0 vote134 views1 answer
Scipy version check
Python
0 vote162 views1 answer
Print to the text file using inline file argument
Python
0 vote124 views1 answer
Typehinting with when parameter has default
Python
2 votes175 views2 answers
How to check django rest framework version
Python Django