Scenario: High-Scale Session Dwell Time Analytics

How can we compute the total active engagement time per user session by summing page view durations capped at 10 minutes?

Production⏱ 3 Minutes⭐ 870 XP

🤔 THE QUESTION

How can we compute the total active engagement time per user session by summing page view durations capped at 10 minutes?

💡 WHAT IS IT?

Computing pageview intervals with lag(), capping outlier intervals with least(), and aggregating sums active dwell time.

🎯 WHAT IS IT USED FOR?

Media publisher analytics calculating true content engagement and ad viewability metrics.

💻 EXAMPLE

w = Window.partitionBy("session_id").orderBy("view_time")
df_dwell = df_views.withColumn("next_time", lead("view_time", 1).over(w)).withColumn("raw_duration", col("next_time").cast("long") - col("view_time").cast("long")).withColumn("duration_sec", when(col("raw_duration").isNull(), 30).otherwise(least(col("raw_duration"), lit(600))))
df_session_totals = df_dwell.groupBy("session_id").agg(sum("duration_sec").alias("total_session_seconds"))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Scenario: High-Scale Session Dwell Time Analytics.

✓ Interval duration math
✓ least() outlier capping
✓ Session engagement aggregation

🚀 Start Mission

← Prev (Scenario: User Retention Cohort Analysis)Next Lesson (Scenario: Dynamic Cross-Currency Price Normalization) →