/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

1 vote131 views1 answer
Matplotlib x axis at the top
Python
1 vote152 views1 answer
Find index of max value in 2d array python
Python
6 votes117 views2 answers
Add python kernel to Jupyter
Python
10 votes117 views1 answer
Print first dictionary keys python
Python
0 vote186 views2 answers
How to subtract minutes from time in python
Python
0 vote110 views2 answers
Odoo wizard current user login
Python
0 vote130 views1 answer
Python selenium login button class click
Python
0 vote136 views1 answer
Plt.imshow typeerror invalid dimensions for image data
Python
0 vote112 views1 answer
Pandas string manipulation on column
Python
6 votes117 views4 answers
How to encrypt text in python
Python