/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

3 votes159 views1 answer
Close terminal python
Python
0 vote132 views1 answer
Keras functional api embedding layer
Python
0 vote140 views1 answer
Find a file in python
Python
1 vote150 views2 answers
Polymorphism in python
Python
1 vote161 views1 answer
How to make a calcukatir in python
Python
9 votes143 views2 answers
Clear terminal in python
Python
8 votes128 views6 answers
Python int to binary
Python
-1 vote130 views1 answer
How to separate numeric and categorical variables in python
Python
0 vote188 views1 answer
Replace by positions a string in a list with another string python
Python
-1 vote145 views1 answer
Print multiplication table python
Python