Skip to main content
PYSPARK • LESSON 55

Substring Extraction with substring()

How do we extract the 4-digit year prefix from an ISO date string using 1-based indexing?

Intermediate2 Minutes340 XP
🤔 THE QUESTION

How do we extract the 4-digit year prefix from an ISO date string using 1-based indexing?

💡 WHAT IS IT?

substring(strCol, pos, len) extracts a sub-string starting at 1-based index pos for len characters.

🎯 WHAT IS IT USED FOR?

Extracting date components, parsing fixed-length transaction identifiers, and area code slicing.

💻 EXAMPLE
from pyspark.sql.functions import col, substring

df = df.withColumn("year_prefix", substring(col("event_date"), 1, 4))

🎯 Mission Objectives

Practice typing production-grade PySpark code for Substring Extraction with substring().

  • Import substring function
  • Specify 1-based start index and length
  • Extract 4-digit year prefix