/Python/Build the union of a list of RDDs

Build the union of a list of RDDs

# Build the union of a list of RDDs

path = os.path.join(tempdir, "union-text.txt")
with open(path, "w") as testFile:
  _ = testFile.write("Hello")
textFile = sc.textFile(path)
textFile.collect()
# ['Hello']
parallelized = sc.parallelize(["World!"])
sorted(sc.union([textFile, parallelized]).collect())
# ['Hello', 'World!']

Source: spark.apache.org

Similar codes

1 vote157 views1 answer
ImportError: dynamic module does not define module export function
Python
0 vote155 views1 answer
Index in for loop
Python
0 vote157 views1 answer
Using django model translation with django rest
Python
0 vote149 views1 answer
Python return min length of list
Python
2 votes166 views2 answers
Python jinja2 loop dict
Python
0 vote159 views1 answer
Python check if false in dict
Python
0 vote160 views1 answer
Scikit learn tree
Python
0 vote156 views1 answer
Use the index of a dataframe for another dataframe
Python
0 vote186 views1 answer
Day 14 scope hackerrank solution in python
Python
4 votes175 views1 answer
Circle python programe
Python