Identifying Missing Data with isNull()

How do we query and inspect records where customer email addresses are missing?

Intermediate⏱ 2 Minutes⭐ 320 XP

🤔 THE QUESTION

How do we query and inspect records where customer email addresses are missing?

💡 WHAT IS IT?

isNull() returns a boolean Column evaluating to True if the referenced field is SQL NULL.

🎯 WHAT IS IT USED FOR?

Data profiling, isolating corrupted ingress payloads, and alerting on missing mandatory fields.

💻 EXAMPLE

from pyspark.sql.functions import col

df.filter(col("email").isNull()).show()

Practice typing production-grade PySpark code for Identifying Missing Data with isNull().