Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

-2 votes169 views4 answers
Merge multiple dataframs
Python
0 vote177 views1 answer
Pygame.event.get() in pygame
Python
0 vote205 views1 answer
To show the individual process IDs involved
Python
0 vote162 views1 answer
Cv2 put font on center
Python
0 vote170 views1 answer
Check if timestamp is NaT
Python
46 votes163 views6 answers
Python integer to string
Python
1 vote113 views1 answer
Flask app route in different files
Python
0 vote176 views1 answer
Email slicer in python code user input
Python
48 votes163 views4 answers
Console clear python
Python
3 votes181 views2 answers
Np load csv
Python