Skip to main content
PYSPARK • LESSON 115

Partition-Level Aggregate Comparison

How do we compute each employee's salary as a percentage share of their department's total payroll?

Advanced2 Minutes640 XP
🤔 THE QUESTION

How do we compute each employee's salary as a percentage share of their department's total payroll?

💡 WHAT IS IT?

Applying sum(salary) over an unordered departmental window and dividing individual salary by the total.

🎯 WHAT IS IT USED FOR?

Payroll budget distribution, cost center allocation, and organizational equity benchmarking.

💻 EXAMPLE
from pyspark.sql.window import Window
from pyspark.sql.functions import col, sum

dept_window = Window.partitionBy("department")
df = df.withColumn("dept_total_salary", sum(col("salary")).over(dept_window)) \
       .withColumn("salary_pct_of_dept", col("salary") / col("dept_total_salary"))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Partition-Level Aggregate Comparison.

  • Compute partition sum without collapsing rows
  • Calculate individual percentage share of department
  • Produce compensation distribution metric