Conditional NULL Handling with when()

How do we conditionally replace null category values with 'Uncategorized' while preserving valid strings?

Intermediate⏱ 2 Minutes⭐ 370 XP

🤔 THE QUESTION

How do we conditionally replace null category values with 'Uncategorized' while preserving valid strings?

💡 WHAT IS IT?

Combining when(col.isNull(), ...) with otherwise(col) to standardize missing category fields.

🎯 WHAT IS IT USED FOR?

Product catalog classification, taxonomy standardization, and reporting dimension cleansing.

💻 EXAMPLE

from pyspark.sql.functions import col, when

df = df.withColumn(
    "normalized_category",
    when(col("category").isNull(), "Uncategorized")
    .otherwise(col("category"))
)

🎯 Mission Objectives

Practice typing production-grade PySpark code for Conditional NULL Handling with when().

✓ Check isNull() within when() condition
✓ Assign fallback string literal
✓ Preserve existing valid categories

🚀 Start Mission

← Prev (First Non-Null Selection with coalesce())Next Lesson (Automated Data Quality Null Audit) →