Skip to main content
PYSPARK โ€ข LESSON 138

Navigating Nested Struct Fields with Dot Notation

How do we project individual sub-fields (like address.city) from a nested struct column using dot notation?

Advancedโฑ 2 Minutesโญ 690 XP
๐Ÿค” THE QUESTION

How do we project individual sub-fields (like address.city) from a nested struct column using dot notation?

๐Ÿ’ก WHAT IS IT?

Referencing sub-fields with col('struct_name.field_name') to query nested struct hierarchies directly.

๐ŸŽฏ WHAT IS IT USED FOR?

Flattening nested records, accessing sub-properties, and extracting specific fields from JSON/Avro payloads.

๐Ÿ’ป EXAMPLE
from pyspark.sql.functions import col

df.select(col("user_id"), col("address.city"), col("address.zipcode")).show()

๐ŸŽฏ Mission Objectives

Practice typing production-grade PySpark code for Navigating Nested Struct Fields with Dot Notation.

  • โœ“ Use dot notation to reference nested fields
  • โœ“ Select specific sub-properties
  • โœ“ Display unpacked address attributes