/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

0 vote131 views1 answer
Python np array get dimantion
Python
5 votes137 views1 answer
Convert jupyter notebook to python cmd line
Python
0 vote149 views1 answer
Boxplot for all columns in python
Python
4 votes128 views1 answer
How to write your first python program
Python
0 vote130 views1 answer
List in one line of text in python
Python
1 vote149 views2 answers
Pytorch view -1 meaning
Python
2 votes120 views1 answer
AttributeError: module 'tensorflow' has no attribute 'random_normal'
Python
0 vote145 views1 answer
Heroku[web.1]: Process exited with status 3
Python
0 vote126 views1 answer
What is norways politics
Python
2 votes97 views1 answer
Python check if email is valid
Python