/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

1 vote148 views1 answer
Message box on closing window event in tkinter
Python
8 votes172 views1 answer
Discord.py making a bot join
Python
1 vote160 views1 answer
Read csv python without id
Python
0 vote156 views1 answer
Starting variable name with underscore python
Python
2 votes150 views1 answer
Python RuntimeWarning: overflow encountered in long_scalars
Python
0 vote179 views1 answer
Django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: auth
Python Django
26 votes165 views2 answers
Python year month day hour minute second
Python
3 votes184 views1 answer
Email authentication python
Python
0 vote173 views1 answer
Remove prefix in python 3.6
Python
-1 vote153 views1 answer
How to create multiple variables in a loop python
Python