Skip to main content
PYSPARK • LESSON 282

Scenario: Real-Time Fraud Event Detection

How can we detect suspicious transaction velocity where a single card is swiped in different cities within 10 minutes?

Production3 Minutes810 XP
🤔 THE QUESTION

How can we detect suspicious transaction velocity where a single card is swiped in different cities within 10 minutes?

💡 WHAT IS IT?

Using lag() to compare transaction timestamps and geographic coordinates identifies impossible physical travel velocity.

🎯 WHAT IS IT USED FOR?

Credit card fraud prevention systems alerting security teams to stolen credentials in real time.

💻 EXAMPLE
w = Window.partitionBy("card_number").orderBy("transaction_time")
df_fraud = df_tx.withColumn("prev_city", lag("city", 1).over(w)).withColumn("prev_time", lag("transaction_time", 1).over(w)).withColumn("time_diff_sec", col("transaction_time").cast("long") - col("prev_time").cast("long")).filter((col("city") != col("prev_city")) & (col("time_diff_sec") < 600))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Scenario: Real-Time Fraud Event Detection.

  • Geographic velocity detection
  • lag() timestamp and location comparison
  • Real-time fraud rule scoring