Check null all column pyspark

Pyspark check all columns for null values

from pyspark.sql.functions import isnan, when, count, col

df.select([count(when(isnan(c), c)).alias(c) for c in df.columns]).show()
+-------+----------+---+
|session|timestamp1|id2|
+-------+----------+---+
|      0|         0|  3|
+-------+----------+---+

Source: stackoverflow.com

Check null all column pyspark

data_indexed.select([count(when(col(c).isNull(),c)).alias(c) for c in data_indexed.columns]).toPandas().T

Similar codes

21 votes129 views1 answer
How to plot a graph using matplotlib
Python
8 votes128 views2 answers
Python get public ip address
Python
0 vote143 views1 answer
True false array to black and white
Python
0 vote127 views1 answer
Python remove second occurrence of character in string
Python
5 votes137 views2 answers
Append string variable with integer python
Python
0 vote131 views1 answer
How to iterate row wise using 2d integer array in python
Python
0 vote176 views1 answer
No module named 'bootstrap4' django
Python Django
0 vote134 views1 answer
Explore data dataframe pandas
Python
1 vote131 views1 answer
Como deixar todas as letras maiusculas no python
Python
10 votes126 views2 answers
Python create file if not exists
Python