Spark Architecture and Execution — Big Data Processing & Lakehouse En…
How a running Spark application actually works
Steps in Spark Architecture and Execution
- Driver, Executors and Cluster Architecture — beginner · How the pieces of a running Spark application fit together
- The Spark UI and Job Monitoring — beginner · Reading the Spark UI to understand what a job is actually doing
- Stages, Tasks and Parallelism — beginner · How Spark divides work and what controls the degree of parallelism
- Wide vs Narrow Transformations — beginner · Why some operations require a shuffle and others don't
- Fault Tolerance in Spark — beginner · How Spark recovers from a failed executor without losing work
- Debugging a Slow Spark Job — beginner · A structured approach to diagnosing performance problems
Part of
- Big Data Processing & Lakehouse Engineering roadmap — the full learning path