Reading CSV Files

How do we load raw CSV customer data from disk into a distributed DataFrame?

Beginner⏱ 2 Minutes⭐ 150 XP

🤔 THE QUESTION

How do we load raw CSV customer data from disk into a distributed DataFrame?

💡 WHAT IS IT?

spark.read.csv() reads text files formatted with comma-separated values into a Spark DataFrame.

🎯 WHAT IS IT USED FOR?

Ingesting raw data feeds, customer exports, and legacy database dump files into the data lake.

💻 EXAMPLE

customers_df = spark.read.csv("data/customers.csv")

Practice typing production-grade PySpark code for Reading CSV Files.