Pyspark rdd method

>>> df.repartition(10).rdd.getNumPartitions()
10
>>> data = df.union(df).repartition("age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+
>>> data = data.repartition(7, "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  2|Alice|
|  5|  Bob|
|  2|Alice|
|  5|  Bob|
+---+-----+
>>> data.rdd.getNumPartitions()
7
>>> data = data.repartition("name", "age")
>>> data.show()
+---+-----+
|age| name|
+---+-----+
|  5|  Bob|
|  5|  Bob|
|  2|Alice|
|  2|Alice|
+---+-----+

Source: spark.apache.org

Similar codes

25 votes101 views4 answers
Tkinter entry
Python
1 vote102 views1 answer
Les librairies python a maitriser pour faire du machine learning
Python
0 vote116 views1 answer
Repeat every entru n times
Python
2 votes107 views1 answer
How to check if an element is visible on the web page in selenium python
Python
91 votes95 views6 answers
Conda create environment python 3
Python
1 vote98 views2 answers
To see version matplotlib
Python
0 vote104 views1 answer
Python3 download file
Python
0 vote108 views1 answer
Python set table widget header
Python
0 vote91 views1 answer
Change heading treeview tkinter
Python
0 vote93 views1 answer
How to save date in cookie Python
Python