Partition Pruning — Roadmap Step & Resources
Letting Spark skip irrelevant data based on partition columns
- Level: beginner
- Estimated time: 1 days
- Roadmap: Big Data Processing & Lakehouse Engineering
Before this step
Study resources
- Databricks Documentation (Article) — Official documentation for the Databricks managed Spark and lakehouse platform. Background reading/viewing for: Partition Pruning.
- Apache Spark Documentation (Article) — Official documentation on partitioning and data layout. Relevant to: Partition Pruning.
Part of
- Partitioning and Data Layout — section
- Big Data Processing & Lakehouse Engineering roadmap — the full learning path