Skip to main content
PYSPARK โ€ข LESSON 115

Partition-Level Aggregate Comparison

How do we compute each employee's salary as a percentage share of their department's total payroll?

Advancedโฑ 2 Minutesโญ 640 XP
๐Ÿค” THE QUESTION

How do we compute each employee's salary as a percentage share of their department's total payroll?

๐Ÿ’ก WHAT IS IT?

Applying sum(salary) over an unordered departmental window and dividing individual salary by the total.

๐ŸŽฏ WHAT IS IT USED FOR?

Payroll budget distribution, cost center allocation, and organizational equity benchmarking.

๐Ÿ’ป EXAMPLE
from pyspark.sql.window import Window
from pyspark.sql.functions import col, sum

dept_window = Window.partitionBy("department")
df = df.withColumn("dept_total_salary", sum(col("salary")).over(dept_window)) \
       .withColumn("salary_pct_of_dept", col("salary") / col("dept_total_salary"))

๐ŸŽฏ Mission Objectives

Practice typing production-grade PySpark code for Partition-Level Aggregate Comparison.

  • โœ“ Compute partition sum without collapsing rows
  • โœ“ Calculate individual percentage share of department
  • โœ“ Produce compensation distribution metric