Dropping Corrupted Rows with dropna()

How do we drop records where critical business keys (order_id or customer_id) are missing?

Intermediate⏱ 2 Minutes⭐ 350 XP

🤔 THE QUESTION

How do we drop records where critical business keys (order_id or customer_id) are missing?

💡 WHAT IS IT?

dropna(subset=[...]) drops rows containing NULL values within the specified subset of columns.

🎯 WHAT IS IT USED FOR?

Enforcing mandatory relational constraints before loading into star-schema data warehouses.

💻 EXAMPLE

df = df.dropna(subset=["order_id", "customer_id"])

Practice typing production-grade PySpark code for Dropping Corrupted Rows with dropna().