Check null all column pyspark

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Check null all column pyspark

data_indexed.select([count(when(col(c).isNull(),c)).alias(c) for c in data_indexed.columns]).toPandas().T

Similar codes

1 vote139 views1 answer
Python check if variable is iterable
Python
1 vote140 views1 answer
Create list integers
Python
0 vote136 views2 answers
How to get current page url in django template
Python
0 vote151 views1 answer
Read emails from gmail python
Python
0 vote158 views1 answer
Flask session auto logout in 5 mins
Python
0 vote140 views1 answer
How to import something in python
Python
65 votes147 views4 answers
How to sum all the numbers in a list in python
Python
6 votes116 views2 answers
Django queryset get all distinct
Python
0 vote157 views3 answers
When was python 3 released
Python
0 vote166 views2 answers
How to get current latitude and longitude in python
Python