Streaming & Real-Time Data Engineering Roadmap — Step-by-step Learnin…
Building real-time data pipelines with Kafka and stream processing engines: event-driven architecture, exactly-once semantics and low-latency systems
The Streaming & Real-Time Data Engineering roadmap
- Streaming Data Fundamentals — beginner
- Event-Driven Architecture Basics — beginner
- Event-Driven Architecture Fundamentals — beginner
- Events vs Messages vs Streams — beginner
- Publish-Subscribe Patterns — beginner
- Event Sourcing Fundamentals — beginner
- Designing Events for a Streaming System — beginner
- Kafka Fundamentals — beginner
- Kafka Fundamentals — beginner
- Kafka's Log-Based Architecture — beginner
- Kafka Clusters and Brokers — beginner
- ZooKeeper vs KRaft — beginner
- Kafka vs Other Messaging Systems — beginner
- Producers and Consumers — beginner
- Kafka Producers — beginner
- Kafka Consumers and Consumer Groups — beginner
- Consumer Offset Management — beginner
- Producer Acknowledgment and Reliability — beginner
- Building a Simple Producer-Consumer Pipeline — beginner
- Kafka Architecture Deep Dive — beginner
- Kafka Cluster Architecture — beginner
- Replication and Fault Tolerance in Kafka — beginner
- Leader Election and Partition Replicas — beginner
- Kafka Storage Internals — beginner
- Kafka Connect for Integration — beginner
- Debugging Kafka Cluster Issues — beginner
- Topics and Partitioning — beginner
- Topic and Partition Design — beginner
- Partition Keys and Data Distribution — beginner
- Partition Rebalancing — beginner
- Ordering Guarantees in Kafka — beginner
- Scaling Partitions Over Time — beginner
- Delivery Guarantees and Semantics — beginner
- At-Most-Once, At-Least-Once and Exactly-Once — beginner
- Idempotent Producers — beginner
- Transactional Messaging in Kafka — beginner
- Handling Duplicate Events Downstream — beginner
- Choosing a Delivery Guarantee for Your Use Case — beginner
- Schema Management and Registries — beginner
- Why Schema Management Matters for Streams — beginner
- Schema Registry Fundamentals — beginner
- Avro, Protobuf and JSON Schema Compared — beginner
- Schema Evolution and Compatibility Rules — beginner
- Enforcing Schema Contracts Across Teams — beginner
- Stream Processing Fundamentals — beginner
- Stream Processing Fundamentals — beginner
- Stateless vs Stateful Stream Processing — beginner
- Stream Processing Frameworks Compared — beginner
- Kafka Streams Fundamentals — beginner
- ksqlDB for SQL-Based Stream Processing — beginner
- Choosing a Stream Processing Framework — beginner
- Apache Flink Fundamentals — beginner
- Apache Flink Fundamentals — beginner
- Flink's DataStream API — beginner
- Flink vs Spark Structured Streaming — beginner
- Flink Job Deployment Models — beginner
- Flink SQL — beginner
- Windowing and Event Time — beginner
- Windowing Fundamentals — beginner
- Tumbling, Sliding and Session Windows — beginner
- Event Time vs Processing Time — beginner
- Watermarks and Late Data — beginner
- Windowed Aggregations in Practice — beginner
- Stateful Stream Processing — beginner
- State in Stream Processing — beginner
- State Backends and Checkpointing — beginner
- Managing Large State — beginner
- State Recovery After Failures — beginner
- Common Stateful Processing Patterns — beginner
- Exactly-Once Processing Guarantees — beginner
- Exactly-Once Processing Fundamentals — beginner
- End-to-End Exactly-Once Pipelines — beginner
- Checkpointing and Barrier Alignment — beginner
- Transactional Sinks — beginner
- Testing for Exactly-Once Behavior — beginner
- Common Pitfalls That Break Exactly-Once Guarantees — beginner
- Streaming Architecture Patterns — beginner
- Lambda Architecture — beginner
- Kappa Architecture — beginner
- Change Data Capture into Streaming Pipelines — beginner
- Streaming ETL Patterns — beginner
- Designing for Backpressure — beginner
- Monitoring and Operating Streaming Systems — beginner
- Monitoring Streaming Pipeline Health — beginner
- Consumer Lag Monitoring — beginner
- Alerting for Streaming Systems — beginner
- Tracing Events Through a Streaming Pipeline — beginner
- Incident Response for Streaming Outages — beginner
- Scaling Streaming Infrastructure — beginner
- Scaling Kafka Clusters — beginner
- Scaling Stream Processing Jobs — beginner
- Multi-Region Streaming Architectures — beginner
- Capacity Planning for Streaming Systems — beginner
- Cost Optimization for Streaming Infrastructure — beginner
- Career and Capstone — advanced