Data Filtering in PySpark DataFrames (Complete Guide with Examples)
Learn how to filter data in PySpark DataFrames using conditions, column expressions, multiple filters, and row extraction with examples and outputs.
8 docs tagged with "Big Data"
View all tagsIntroduction to PySpark — Big Data Processing with Python
Learn what PySpark is, why it matters, and how to process big data, build ETL pipelines, run Spark SQL, and use distributed machine learning with Python.
Joins in PySpark DataFrames (Full Beginner Guide)
Learn all types of joins in PySpark DataFrames — inner, left, right, outer, semi, anti, and cross join with clear examples, code, and explanations.
PySpark Data I/O — Reading, Writing & Optimizing Big Data
Learn how to efficiently read, write, and process data in PySpark including CSV, JSON, Parquet, ORC, JDBC databases, cloud storage, streaming, and compression. A complete guide for beginners and data engineers.
PySpark DataFrame Basics (Part 1) — Complete Beginner Guide
Learn the fundamentals of PySpark DataFrames including creation, schema inspection, show(), describe(), and column operations. Perfect for beginners starting with distributed data processing.
PySpark DataFrame Basics (Part 2) — Custom Schemas, Column Ops & SQL
Learn how to define custom schemas, select columns, add new columns, rename columns, inspect types, and run SQL queries on PySpark DataFrames.
PySpark Setup and Configuration — Simple Guide
Step-by-step guide to installing, setting up, and configuring PySpark for local and cluster environments. Learn SparkSession initialization, environment variables, and configuration best practices.
Working with Dates and Timestamps in PySpark DataFrames (Full Guide)
Complete guide to date and timestamp operations in PySpark, including extracting date components, aggregations, ratios, and SQL queries with real examples.