/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

0 vote293 views1 answer
OpenPyXl Get Sheet Names
Python
-1 vote142 views1 answer
How to subtract dates in Python
Python
0 vote145 views1 answer
Pytest runtimeerror: no application found. either work inside a view function or push an application context
Python
2 votes172 views1 answer
Django template capitalize equivalent
Python Django
0 vote140 views1 answer
How to define an empyt dic tin python
Python
0 vote129 views1 answer
Python read binary
Python
0 vote186 views3 answers
Dataset analysis in python photo photoelectric effect
Python
0 vote135 views1 answer
Python first
Python
1 vote138 views1 answer
How to sort dict by value
Python
0 vote151 views1 answer
Non blocking socket python
Python