/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

6 votes185 views2 answers
How to limit a command to a permission in discord.py
Python
18 votes188 views5 answers
Get list with random numbers python
Python
11 votes253 views5 answers
Linear regression in python
Python
0 vote182 views1 answer
How to test if a webpage is an image python requests
Python
0 vote244 views1 answer
How to create a spark schema using a string
Python
0 vote234 views1 answer
Import "sendgrid" could not be resolved django
Python Django
0 vote271 views1 answer
Find char in str python
Python
134 votes153 views15 answers
Ternary operator python
Python
23 votes190 views2 answers
Flask or django
Python
19 votes209 views2 answers
Learn python 3
Python