Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

0 vote181 views1 answer
How to make one list from nested list
Python
0 vote149 views1 answer
How to copy content of one file to another in python
Python
36 votes177 views6 answers
Move file python
Python
0 vote165 views1 answer
Change title size matplotlib
Python
8 votes235 views2 answers
Lambda function with if elif else python
Python
2 votes164 views1 answer
Input float python
Python
1 vote155 views1 answer
How to store a number in a variable python
Python
0 vote133 views1 answer
Assosciate keys as list to values in python
Python
0 vote181 views1 answer
Python sort class by attribute
Python
0 vote170 views1 answer
Http.client.invalidurl: url can't
Python