PYSPARK ACADEMY • MODULE 32File & Storage Engineering
Tune Parquet compression codecs, manage multi-level partition hierarchies, apply dynamic partition overwrites, solve the small file problem, and bucket tables.
Expert📚 10 Lessons⭐ 0 / 8450 XP⏳ In Progress
0 / 10 Lessons Completed (0%) LESSON 261🔓 Optimized Parquet Compression Codecs
⏱ 3 Mins • ⭐ 800 XP • Expert
LESSON 262🔒 Multi-Level Hive-Style Partitioning
⏱ 3 Mins • ⭐ 810 XP • Expert
LESSON 263 🔒 Dynamic Partition Overwrite Mode
⏱ 3 Mins • ⭐ 820 XP • Expert
LESSON 264🔒 Controlling File Sizing with maxRecordsPerFile
⏱ 3 Mins • ⭐ 830 XP • Expert
LESSON 265🔒 Solving the Small File Problem via Compaction
⏱ 3 Mins • ⭐ 840 XP • Expert
LESSON 266🔒 Reading and Writing Compressed JSON Payloads
⏱ 3 Mins • ⭐ 850 XP • Expert
LESSON 267🔒 Reading with Schema Evolution & mergeSchema
⏱ 3 Mins • ⭐ 860 XP • Expert
LESSON 268🔒 Bucketing Data with bucketBy() and sortBy()
⏱ 3 Mins • ⭐ 870 XP • Expert
LESSON 269🔒 Reading Cloud Storage with Recursive File Lookup
⏱ 3 Mins • ⭐ 880 XP • Expert
LESSON 270🔒 Production Partitioned Lakehouse Export Pipeline
⏱ 3 Mins • ⭐ 890 XP • Expert