Skip to main content
PYSPARK • LESSON 67

Imputing Defaults with fillna()

How do we replace missing numeric and string values with custom default values using a dictionary?

Intermediate2 Minutes340 XP
🤔 THE QUESTION

How do we replace missing numeric and string values with custom default values using a dictionary?

💡 WHAT IS IT?

fillna() substitutes NULL entries with specified constants, either across all columns or using a column-to-value map.

🎯 WHAT IS IT USED FOR?

Imputing missing features for machine learning and ensuring reporting dashboards display clean fallback labels.

💻 EXAMPLE
df = df.fillna({"age": 0, "status": "UNKNOWN"})

🎯 Mission Objectives

Practice typing production-grade PySpark code for Imputing Defaults with fillna().

  • Pass dictionary mapping to fillna()
  • Impute default age and status
  • Cleanse missing values in-place