Business Dimension Summary Pipeline

How do we aggregate headcount and annualized budget per department and filter for established teams?

Intermediate⏱ 3 Minutes⭐ 490 XP

🤔 THE QUESTION

How do we aggregate headcount and annualized budget per department and filter for established teams?

💡 WHAT IS IT?

A complete dimensional aggregation pipeline grouping by department, computing annual budgets, and filtering.

🎯 WHAT IS IT USED FOR?

Enterprise financial planning, departmental cost allocation, and resource governance.

💻 EXAMPLE

from pyspark.sql.functions import col, countDistinct, round, sum

dept_metrics_df = df.groupBy("department_id") \
    .agg(
        countDistinct(col("employee_id")).alias("headcount"),
        round(sum(col("monthly_salary")) * 12, 2).alias("annual_budget")
    ) \
    .filter(col("headcount") >= 5)
dept_metrics_df.show()

🎯 Mission Objectives

Practice typing production-grade PySpark code for Business Dimension Summary Pipeline.

✓ Group by department ID
✓ Calculate distinct headcount and annual budget
✓ Filter departments with at least 5 staff

🚀 Start Mission

← Prev (Conditional Aggregations per Group)Next Lesson (Inner Join Operations) →