Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

0 vote163 views1 answer
How to create api for get data in django rest framework
Python Django
0 vote111 views1 answer
ValueError: unknown is not supported in sklearn.RFECV
Python
0 vote146 views1 answer
Read api from django
Python Django
0 vote87 views2 answers
Tkinter disable resize
Python
5 votes118 views1 answer
Python infinite value
Python
0 vote157 views1 answer
Python compressed for concatenate string
Python
0 vote127 views1 answer
Networkx - unique combinations of paths
Python
1 vote110 views1 answer
Read pdf py
Python
0 vote118 views1 answer
Square all elements in list python
Python
0 vote120 views1 answer
Django form field class
Python