Tim Berglund

An Introduction to Apache Pinot

Tuesday, 3:00 PM EST

When things get a little bit cheaper, we buy a little bit more of them. When things get cheaper by several orders of magnitude, you don't just see changes in the margins, but fundamental transformations in entire ecosystems. Apache Pinot is a driver of this kind of transformation in the world of real-time analytics.

Pinot is a real-time, distributed, user-facing analytics database. The rich set of indexing strategies makes it a perfect fit for running highly concurrent queries on multi-dimensional data, often with millisecond latency. It has out-of-the box integration with Apache Kafka, S3, Presto, HDFS, and more. And it's so much faster on typical analytics workloads that it is not just a marginally better data warehouse, but the cornerstone of the next revolution in analytics: systems that expose data not just to internal decision makers, but to customers using the system itself. Pinot helps expand the definition of a “decision-maker” not just down the org chart, but out of the organization to everyone who uses the system.

In this talk, you'll learn how Pinot is put together and why it performs the way it does. You'll leave knowing its architecture, how to query it, and why it's a critical infrastructure component in the modern data stack. This is a technology you're likely to need soon, so come to this talk for a jumpstart.

Building a Real-Time Analytics Database: a Choose Your Own Adventure Journey

Tuesday, 5:00 PM EST

Have you ever stopped to think about how to build a database? The thing is, there isn't just one way, as we can see by the massive number of data infrastructure options we have to choose from. It's a nonstop series of tradeoffs, each motivated by the constraints the database wants to satisfy. An in-memory transactional database would be one thing. A general-purpose, single-server relational database would be another. A low-latency, horizontally scalable analytics database would be…the journey we're going to take.

In this talk, we'll start by picking a data model, make decisions about serialization and storage, choose indexing strategies, pick a query language, and figure out how to scale, eventually ending up with something that looks remarkably like Apache Pinot, a real-time analytics database. Pinot was built on a journey like this, always optimized for ultra low-latency, user-facing analytics at scale. In the real world, Pinot is used by applications like LinkedIn and UberEats to expose the state of the system not just to internal decision-makers, but to the users of the system itself, including all of us people who consumers of analytical queries. By focusing on the internals of Pinot and the tradeoffs made along the way to build a database of its kind, we'll see how it enables a new class of applications that every user of a system into a decision maker.

Event-Driven Architecture Done Right: Deep Dive

Wednesday, 9:00 AM EST

Event-driven architectures are not new, but they are newly ascendant. For the first time since the client-server revolution of 40 years ago, a new architectural paradigm is changing the way we build systems. Apache Kafka and microservices are at the center of this movement.

In this workshop, we’ll discuss the issues that arise turning a monolith into a set of reactive services, including issues like data contracts, integrating with the systems you can't change, handling request-response interfaces, and more. We'll also discuss common infrastructure choices like Apache Flink and Apache Pinot. Hands-on exercises will focus on understanding your organization's data and forming a plan to refactor that monolith that seems like it will never go away.

Decision Making in Software Teams

Thursday, 11:00 AM EST

Alistair Cockburn has described software development as a game in which we choose among three moves: invent, decide, and communicate. Most of our time at No Fluff is spent learning how to be better at inventing. Beyond that, we understand the importance of good communication, and take steps to improve in that capacity. Rarely, however, do we acknowledge the role of decision making in the life of software teams, what can cause it to go wrong, and how to improve it.

In this talk, we will explore decision making pathologies and their remedies in individual, team, and organizational dimensions. We'll consider how our own cognitive limitations can lead us to to make bad decisions as individuals, and what we might do to compensate for those personal weaknesses. We'll learn how a team can fall into decisionmaking dysfunction, and what techniques a leader might employ to healthy functioning to an afflicted group. We'll also look at how organizational structure and culture can discourage quality decision making, and what leaders to swim against the tide.

Software teams spend a great deal of time making decisions that place enormous amounts of capital on the line. Team members and leaders owe it to themselves to learn how to make them well.