Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote141 views1 answer
Program fibonacci series number in python
Python
0 vote143 views1 answer
Multiclass.roc plot title
Python
0 vote162 views1 answer
How to create a scoreboard for the top 5 players in python
Python
0 vote159 views4 answers
Craeting a method that can take any number of arguments in python
Python
10 votes144 views1 answer
Download videos "hotmart" javascript github
Python
123 votes152 views6 answers
Input python
Python
1 vote150 views5 answers
Transpose a matrix in python
Python
0 vote161 views1 answer
See python function details in vscode
Python
2 votes142 views1 answer
Display python 001
Python
0 vote159 views1 answer
Return tuples form functions in Python
Python