Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

4 votes255 views1 answer
Django slug int url mapping
Python Django
0 vote214 views1 answer
Return a tuple c++ python 3
Python
5 votes201 views2 answers
Python yaml parser
Python
1 vote216 views2 answers
Delete the node at a given position 2 in a linked list and return a reference to the head node. The head is at position 0. The list may be empty after you delete the node. In that case, return a null value.
Python
0 vote203 views1 answer
Python convert latitude longitude to x y
Python
14 votes217 views3 answers
How to get user id discord.py
Python
145 votes231 views5 answers
How to generate a random number python
Python
4 votes185 views1 answer
Pandas date missing from datetime
Python
0 vote198 views1 answer
How to call any function with it name as a string
Python
0 vote199 views1 answer
Xmlrpc get all posts
Python