/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

1 vote103 views1 answer
Python get current process id
Python
0 vote193 views1 answer
Django customize the user model
Python Django
0 vote149 views1 answer
Yahoo! Finance pyhton
Python
0 vote155 views1 answer
This branch is for urllib3 v1.22 and later
Python
95 votes169 views6 answers
Pandas dataframe sort by column
Python
0 vote141 views1 answer
Django amzon like app
Python
0 vote148 views1 answer
Python declare immutable variable
Python
1 vote146 views1 answer
For loop django template count
Python
1 vote141 views1 answer
Camera lags when using with opencv
Python
3 votes159 views2 answers
Matplotlib log scale y axis base
Python