Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote140 views1 answer
Pyqt5 image center
Python
0 vote110 views1 answer
Python move all txt files
Python
0 vote150 views2 answers
How to decrease size of graph in plt.scatter
Python
0 vote150 views1 answer
Balancing paranthesis python
Python
1 vote146 views2 answers
Progress bar python text
Python
0 vote144 views1 answer
Counter python time complexity
Python
4 votes149 views1 answer
Get number of missing values dataframe
Python
16 votes153 views6 answers
Print() in python
Python
14 votes153 views5 answers
Calculate days between two dates python
Python
0 vote145 views1 answer
How to get rid of all null values in array python
Python