Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

1 vote91 views1 answer
How to put in code to download discord py
Python
5 votes110 views7 answers
Programming
Python
3 votes116 views2 answers
Find frequency of numbers in list python
Python
0 vote55 views1 answer
Python second max in numpy array
Python
0 vote149 views1 answer
Pandas df represent a long column name with short name
Python
2 votes60 views3 answers
Label encoding column pandas
Python
1 vote96 views1 answer
Filter dict
Python
1 vote93 views1 answer
Brz: ERROR: TypeError: a bytes-like object is required, not 'str'
Python
12 votes100 views2 answers
Column.replace
Python
3 votes100 views1 answer
Print items in object python
Python