Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

5 votes198 views3 answers
Pandas show duplicate rows
Python
76 votes203 views6 answers
For loop in python
Python
8 votes194 views1 answer
Python make a random number
Python
0 vote172 views1 answer
Check is symmetric python
Python
1 vote219 views1 answer
How to play sound after pressing a button in tkinter
Python
0 vote168 views1 answer
Load text file line in listbox python tkinter site:stackoverflow.com
Python
4 votes190 views2 answers
Change django admin title
Python
0 vote176 views1 answer
Openpyxl add worksheet
Python
2 votes278 views1 answer
Django template capitalize equivalent
Python Django
0 vote203 views3 answers
Print out session information django
Python