Master PySpark and SQL fundamentals through 50 hands-on coding challenges. Learn real-world DataFrame operations, multi-format file ingestion, predicates, deduplication, transformations, and aggregations with interactive Databricks-style notebooks and automated test validation.
Learn how Spark creates, reads and manages DataFrames with catalog tables, schema inspection, count metrics, and lazy evaluation.
Ingest CSV, JSON, and Parquet data, handle custom delimiters, and register session-scoped temporary views.
Project fields with string names, col() functions, attribute access, bracket indexing, alias(), and selectExpr().
Filter records using single and compound boolean predicates (&, |), negation (~), isin(), between(), and date ranges.
Detect missing values with isNull() and isNotNull(), drop incomplete rows via dropna(), and impute defaults with fillna().
Sort records ascending and descending, perform multi-column sorting, limit outputs, and remove duplicates with distinct() and dropDuplicates().
Derive new columns, rename fields with withColumnRenamed(), drop unused columns, cast types, and normalize string casing.
Apply when/otherwise conditional branches, compute summary metrics with agg(), group by department, and join dimension tables.