Mastering the Art of Streaming Infrastructure

Designing a distributed system architecture can be a daunting task, with contradictory requirements and constraints constantly at play. The CAP theorem that directly states the challenges in distributed data stores presents a classic example where developers must choose between consistency, availability, and partition tolerance. The same applies to streaming infrastructure systems, where optimizing for one aspect can come at the cost of another. With cost, throughput, accuracy, and latency as the main constraints for streaming systems, it's crucial to make informed decisions that align with your business goals.

In this session, you'll gain valuable insights into how your system design choices impact your system overall capabilities. You'll also learn about the differences between Flink Streaming and Spark Streaming, both conceptually and in practice. Lastly, you'll understand how combining multiple solutions can be beneficial for your team and business. Join to learn more about the cumbersome world of distributed stream processing systems.