Skip to main content
DatabricksPySparkSnowflakeApache Airflow
Code Keys/PySpark Academy/Module 34/Lesson 290
PYSPARK • LESSON 290

Scenario: Enterprise Master Data Golden Record Creation

How can we merge disparate customer records across Salesforce, Stripe, and Zendesk to create a single Golden Record?

Production⏱ 3 Minutes⭐ 890 XP
🤔 THE QUESTION

How can we merge disparate customer records across Salesforce, Stripe, and Zendesk to create a single Golden Record?

💡 WHAT IS IT?

Using coalesce() across prioritized source columns merges attributes to construct the most complete entity record.

🎯 WHAT IS IT USED FOR?

Master Data Management (MDM) platforms resolving fragmented customer profiles into canonical entities.

💻 EXAMPLE
df_golden = df_merged.withColumn("canonical_name", coalesce(col("salesforce_name"), col("stripe_name"), col("zendesk_name"))).withColumn("canonical_email", coalesce(col("stripe_email"), col("salesforce_email"), col("zendesk_email")))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Scenario: Enterprise Master Data Golden Record Creation.

  • ✓ Golden record creation
  • ✓ Prioritized coalesce() across systems
  • ✓ Enterprise MDM consolidation
🚀 Start Mission
← Prev (Scenario: Dynamic Cross-Currency Price Normalization)Next Lesson (Capstone Part 1: Schema Enforcement on Dirty JSON Payloads) →