/Python/Pyspark check all columns for null values

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Similar codes

1 vote110 views1 answer
Cprofile usage python
Python
4 votes110 views3 answers
Display prime numbers between two intervals in python
Python
3 votes115 views2 answers
To extract out only year month from a date column in pandas
Python
5 votes142 views4 answers
OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
Python
2 votes125 views1 answer
Np.select with multiple conditions
Python
0 vote107 views1 answer
Copy element dynamo revit
Python
0 vote110 views1 answer
Tensorflow_version
Python
1 vote121 views2 answers
Python print utf-8
Python
1 vote128 views1 answer
Erreur install pyaudio
Python
0 vote113 views1 answer
Pytest skip
Python