Decorating Functions with @udf

How do we decorate a Python phone sanitization function with @udf and explicit StringType return type?

Expert⏱ 2 Minutes⭐ 760 XP

🤔 THE QUESTION

How do we decorate a Python phone sanitization function with @udf and explicit StringType return type?

💡 WHAT IS IT?

The @udf decorator converts a Python function into a Spark Column User-Defined Function.

🎯 WHAT IS IT USED FOR?

Applying complex custom Python parsing rules directly across DataFrame columns.

💻 EXAMPLE

from pyspark.sql.functions import udf
from pyspark.sql.types import StringType

@udf(returnType=StringType())
def sanitize_phone(phone):
    return "".join(c for c in phone if c.isdigit()) if phone else None

🎯 Mission Objectives

Practice typing production-grade PySpark code for Decorating Functions with @udf.

✓ Import udf decorator and StringType
✓ Decorate Python cleaning function
✓ Enforce explicit return type

🚀 Start Mission

← Prev (Writing Pure Python Helper Functions)Next Lesson (Explicit Return Types with Lambda UDFs) →