/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote208 views1 answer
Django create username and password from csv
Python Django
0 vote215 views1 answer
Take first n row of dictionary python
Python
13 votes165 views3 answers
Python package version
Python
0 vote182 views1 answer
Listview
Python
1 vote162 views1 answer
Pip fuzzywuzzy
Python
0 vote181 views1 answer
Stutter function in python
Python
15 votes178 views6 answers
Flask install
Python
1 vote234 views1 answer
Procfile flask
Python Flask
7 votes253 views1 answer
How to find the location of a character in a string in python
Python
2 votes183 views1 answer
How to set pyautogui at certain speed
Python