Skip to main content
DatabricksPySparkSnowflakeApache Airflow
Code Keys/PySpark Academy/Module 08/Lesson 58
PYSPARK • LESSON 58

Injecting Current Timestamp with current_timestamp()

How do we record the exact microsecond timestamp when records are processed by Spark?

Intermediate⏱ 2 Minutes⭐ 310 XP
🤔 THE QUESTION

How do we record the exact microsecond timestamp when records are processed by Spark?

💡 WHAT IS IT?

current_timestamp() returns the current date and time as a TimestampType Column.

🎯 WHAT IS IT USED FOR?

Micro-batch stream auditing, SLA latency measurement, and event provenance tracking.

💻 EXAMPLE
from pyspark.sql.functions import current_timestamp

df = df.withColumn("processed_at", current_timestamp())

🎯 Mission Objectives

Practice typing production-grade PySpark code for Injecting Current Timestamp with current_timestamp().

  • ✓ Import current_timestamp function
  • ✓ Add precision timestamp Column
  • ✓ Record pipeline processing time
🚀 Start Mission
← Prev (Injecting Current Date with current_date())Next Lesson (Parsing Date Strings with to_date()) →