/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote159 views1 answer
12000000/12
Python
0 vote136 views1 answer
How to get location using python
Python
0 vote131 views1 answer
Convert 1 to "one" python
Python
0 vote137 views1 answer
Scipy get frequencies of image
Python
0 vote138 views1 answer
Python use getcontext
Python
5 votes222 views4 answers
Xticks and yticks matplotlib
Python
0 vote167 views1 answer
Django create ap
Python Django
0 vote171 views2 answers
Zbarge / s3gui
Python
0 vote154 views1 answer
Python flask how to remove last character from string
Python Flask
4 votes192 views4 answers
Filter django or
Python Django