/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote114 views1 answer
Pandas replace last cell
Python
0 vote106 views1 answer
Root = tk() python 3
Python
0 vote108 views1 answer
Linear congruential generator in python
Python
0 vote108 views1 answer
Torch.utils.data.random_split(dataset, lengths)
Python
0 vote104 views1 answer
Django generate openapi schema command line
Python
9 votes120 views4 answers
Capitalize first letter of each word python
Python
1 vote129 views1 answer
Python custom class indexing
Python
2 votes143 views1 answer
Setting a condition for perfect square in python
Python
0 vote111 views10 answers
Python
Python
1 vote119 views2 answers
Python listen to keyboard input linux
Python