Skip to main content
DatabricksPySparkSnowflakeApache Airflow
Code Keys/PySpark Academy/Module 20/Lesson 154
PYSPARK • LESSON 154

Scalar Pandas UDF with Python Type Hints

How do we define a vectorized scalar Pandas UDF with Python Series type hints to compute sales tax?

Expert⏱ 2 Minutes⭐ 820 XP
🤔 THE QUESTION

How do we define a vectorized scalar Pandas UDF with Python Series type hints to compute sales tax?

💡 WHAT IS IT?

Decorating a function taking pd.Series -> pd.Series with @pandas_udf('double') for vectorized batch operations.

🎯 WHAT IS IT USED FOR?

Vectorized arithmetic, NumPy mathematical functions, and high-speed financial calculations.

💻 EXAMPLE
import pandas as pd
from pyspark.sql.functions import pandas_udf

@pandas_udf("double")
def add_sales_tax(amounts: pd.Series) -> pd.Series:
    return amounts * 1.0825

🎯 Mission Objectives

Practice typing production-grade PySpark code for Scalar Pandas UDF with Python Type Hints.

  • ✓ Define vectorized scalar Pandas UDF
  • ✓ Specify Python type hints (pd.Series -> pd.Series)
  • ✓ Execute vectorized tax computation
🚀 Start Mission
← Prev (Introduction to Vectorized pandas_udf)Next Lesson (Grouped Map Pandas UDF Concepts) →