/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

1 vote144 views1 answer
Check how many times a substring appears in a string
Python
1 vote144 views1 answer
How to convert tuple into list in python
Python
1 vote172 views3 answers
Accessing data on django sessionstore
Python Django
0 vote168 views1 answer
Python arithmetic operation with list
Python
2 votes136 views1 answer
Number to list in python
Python
12 votes166 views2 answers
How to read input from stdin in python
Python
0 vote138 views1 answer
A = [ int(i) for i in range(100, 104)] list python
Python
0 vote147 views1 answer
Python http handler iis
Python
0 vote143 views1 answer
Validating credit card numbers
Python
0 vote146 views1 answer
Jupyter notebook not opening
Python