TimBerglund
Tim serves as the VP of Developer Relations at IBM, where he and his team work to make certain parts of the IBM portfolio more accessible to developers. He is a regular speaker at conferences and a presence on YouTube explaining complex technology topics in an accessible way. He lives with his wife in Arvada, CO. He has three grown children, three step-children, and four grandchildren, with a fifth on the way.
On the inside, Kafka is schemaless, but there is nothing schemaless about the worlds we live in. Our languages impose type systems, and the objects in our business domains have fixed sets of properties and semantics that must be obeyed. Pretending that we can operate without competent schema management does us no good at all.
It has become at truism in the past decade that building systems at scale, using non-relational databases, requires giving up on the transactional guarantees afforded by the relational databases of yore, ACID transactional semantics are fine, but we all know you can’t have them all in a distributed system. Or can we?
Harold McMillan was Prime Minister of England from 1957 to 1963, the last British PM born during Queen Victoria’s rule, and one whose wit and even-keeled nature defined his administration. When asked by a reporter what might force his government off the course he had firmly laid out for it, he allegedly replied “Events, dear boy, events.” The same might be said about what is driving software architectures today. Event-driven systems have enabled organizations to build substantial microservices ecosystems with all of the decoupling and evolvability that we were promised by the distributed computing technologies of 20 years ago. But these systems raise some interesting questions: if events now rule, what has become of entities? If we store events in logs, do we still need databases? Can we merely produce immutable events to trivially scalable logs and loose our microservices to consume them with no regard for what is actually out there in the world?
Developers and architects are increasingly called upon to solve big problems, and we are able to draw on a world-class set of open source tools with which to solve them. Problems of scale are no longer consigned to the web’s largest companies, but are increasingly a part of ordinary enterprise development. At the risk of only a little hyperbole, we are all distributed systems engineers now.
Kafka has become a key data infrastructure technology, and we all have at least a vague sense that it is a messaging system, but what else is it? How can an overgrown message bus be getting this much buzz? Well, because Kafka is merely the center of a rich streaming data platform that invites detailed exploration.
In a world increasingly defined in software, is the database–a tool primarily built to aid human-computer interaction–always the right tool to choose? In this talk, we’ll look at a new type of database, built not only for the tables and columns we’re familiar with, but also the continuous, never-ending "streams of events" that represent data as it moves. We’ll take a look at ksqlDB’s syntax and show how it can replace bespoke Kafka Consumers with short, declarative queries.
Zero fluff.