/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

0 vote134 views1 answer
Sum of positive numbers in array with negative python
Python
0 vote129 views1 answer
Visualizzare csv in pycharm pandas read_csv
Python
2 votes132 views1 answer
How to create a loop in python turtle
Python
2 votes148 views1 answer
How to save query data into dataframe pscopg2
Python
0 vote129 views1 answer
Autopy in python install
Python
0 vote180 views1 answer
Replace by positions a string in a list with another string python
Python
0 vote121 views1 answer
Python sklearn knn regression example
Python
0 vote251 views5 answers
Django-tool-bar
Python Django
5 votes116 views2 answers
Pandas create a new column based on condition of two columns
Python
0 vote121 views1 answer
Make exact copy of an environment
Python