Conditional Aggregation with when()

How can we aggregate metrics conditionally within a groupBy operation without writing multiple subqueries?

Advanced⏱ 3 Minutes⭐ 830 XP

🤔 THE QUESTION

How can we aggregate metrics conditionally within a groupBy operation without writing multiple subqueries?

💡 WHAT IS IT?

Wrapping when() inside sum() or count() allows selective aggregation of rows meeting specific business criteria.

🎯 WHAT IS IT USED FOR?

Calculating KPI matrices such as completed vs failed orders in a single aggregation step.

💻 EXAMPLE

df_metrics = df.groupBy("region").agg(sum(when(col("status") == "COMPLETED", col("amount")).otherwise(0)).alias("completed_revenue"), count(when(col("status") == "FAILED", 1)).alias("failed_count"))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Conditional Aggregation with when().

✓ sum(when()) pattern
✓ Selective metric calculation
✓ Single-pass pivot aggregation

🚀 Start Mission

← Prev (Multiple Expression Pipeline)Next Lesson (Dynamic Column Transformations) →