Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

0 vote136 views1 answer
X not defined python
Python
0 vote135 views1 answer
Lsit append in python
Python
0 vote106 views1 answer
Pandas count unique index
Python
14 votes128 views6 answers
Python if condition
Python
0 vote160 views1 answer
How to cerate a bar chart seaborn
Python
0 vote135 views1 answer
How to read unicode in python
Python
39 votes142 views5 answers
Csv to python
Python
9 votes130 views1 answer
Cv2 grayscale
Python
0 vote136 views1 answer
Python enumerate()
Python
2 votes131 views2 answers
How to code a yes or no question in python v3.8
Python