/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

13 votes101 views2 answers
How to copy one dictionary to another in python
Python
0 vote97 views2 answers
Django rest framework authentication
Python
3 votes244 views2 answers
List images in directory python
Python
2 votes106 views1 answer
Django template capitalize equivalent
Python
1 vote112 views1 answer
Python convert to percentage
Python
0 vote108 views1 answer
How to make an entire dataframe show in jupyter
Python
1 vote110 views2 answers
Python not null
Python
1 vote153 views1 answer
Module 'torch' has no 'stack' memberpylint(no-member)
Python
0 vote108 views1 answer
Pygame point at mouse
Python
2 votes65 views2 answers
Python telegram bot send message
Python