Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

0 vote142 views1 answer
Leetcode 206 python
Python
0 vote143 views1 answer
Python set match two list
Python
1 vote173 views2 answers
How to remove the very last character of a text file in python
Python
0 vote144 views1 answer
Django password field encryption
Python
2 votes141 views1 answer
Filter function using lambda in python
Python
1 vote164 views1 answer
Create new column using dictionary padnas
Python
2 votes136 views1 answer
Torch device
Python
0 vote141 views1 answer
Python - Cómo Jugar archivo Mp3
Python
1 vote153 views1 answer
Frequency
Python
6 votes115 views2 answers
Django queryset get all distinct
Python