/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote181 views3 answers
Discord.py delete own message
Python
0 vote137 views1 answer
Imagedatagenerator split train test
Python
0 vote186 views1 answer
Polycarp and Coins codeforces solution
Python
2 votes209 views2 answers
Python split string capital letters
Python
0 vote165 views1 answer
Installing python3.9 on linux mint 20
Python
14 votes174 views1 answer
Python cheat sheet pdf download
Python
0 vote199 views1 answer
Tb to pb with python calculator
Python
1 vote181 views2 answers
Python sizeof
Python
0 vote174 views1 answer
Standardise columns python
Python
0 vote185 views1 answer
Pandas select only columns with na
Python