Starting Streaming Queries with writeStream

How do we start a streaming execution query outputting micro-batches to the console sink for debugging?

Production⏱ 2 Minutes⭐ 1120 XP

🤔 THE QUESTION

How do we start a streaming execution query outputting micro-batches to the console sink for debugging?

💡 WHAT IS IT?

df.writeStream.format('console').start() kicks off background micro-batch processing to the console sink.

🎯 WHAT IS IT USED FOR?

Local stream testing, inspecting real-time transformations, and debugging streaming micro-batches.

💻 EXAMPLE

query = streaming_df.writeStream \
    .format("console") \
    .start()

Practice typing production-grade PySpark code for Starting Streaming Queries with writeStream.