Brian Sletten

Building Production ML/AI Systems

Tuesday, 1:00 PM EST

On the one hand, Machine Learning (ML) and AI Systems are just more software and can be treated as such from our development efforts. On the other hand, they behave very differently and our capacity to test, verify, validate, and scale them requires a different set of perspectives and skills.

This presentation will walk you through some of these unexpected differences and how to plan for them. No specific background in ML/AI is required, but you are encouraged to be generally aware of these fields. The AI Crash Course would be a good start.

We will cover:

Matching Capabilities to Needs
Performance Tuning
Vector Databases
Testing Strategies
MLOPs/AIOps Techniques
Evolving these Systems Over Time

Building Production ML/AI Systems

Tuesday, 3:00 PM EST

On the one hand, Machine Learning (ML) and AI Systems are just more software and can be treated as such from our development efforts. On the other hand, they behave very differently and our capacity to test, verify, validate, and scale them requires a different set of perspectives and skills.

This presentation will walk you through some of these unexpected differences and how to plan for them. No specific background in ML/AI is required, but you are encouraged to be generally aware of these fields. The AI Crash Course would be a good start.

We will cover:

Matching Capabilities to Needs
Performance Tuning
Vector Databases
Testing Strategies
MLOPs/AIOps Techniques
Evolving these Systems Over Time

Automating Security Fixes with OpenRewrite: Patching Vulnerabilities Across the Codebase

Tuesday, 5:00 PM EST

Security problems empirically fall into two categories: bugs and flaws. Roughly half of the problems we encounter in the wild are bugs and about half are design flaws. A significant number of the bugs can be found through automated testing tools which frees you up to focus on the more pernicious design issues. 

 In addition to detecting the presence of common bugs, however, we can also imagine automating the application of corrective refactoring. In this talk, I will discuss using OpenRewrite to fix common security issues and keep them from coming back.

In this talk we will focus on:

Using OpenRewrite to automatically identify and fix known security vulnerabilities.
Integrating security scans with OpenRewrite for continuous improvement.
*Free up your time to address larger concerns by addressing the pedestrian but time-consuming security bugs.

RAG Systems : Using LLMs with Guardrails

Wednesday, 9:00 AM EST

Large Language Models (LLMs) such as ChatGPT and Llama have impressed us with what they can do. They have also horrified us with what they actually do when they are employed with no protection: hallucinations, stale knowledge bases, no conceptual basis for reasoning, and a capacity for toxic and inappropriate content generation. Rather than avoid them altogether or risk legal liability or brand damage, we can put some guardrails around them to benefit from their best traits without fearing their worst.

Retrieval Augmented Generation (RAG) systems augment the process to make it behave more to our liking. Come hear what you can do to benefit from AI systems without fearing them.

We will cover examples using LangChain and LlamaIndex, two open source frameworks for working with LLMs and creating RAG infrastructure.

We will cover:

Introduction to LLMs
Risks and Limitations
Basic RAG Systems
Embeddings
Vector Databases
Prompt Engineering
Testing and Validating LLMs and RAG Systems
Advanced Techniques
AI as Judge

RAG Systems : Using LLMs with Guardrails

Wednesday, 11:00 AM EST

Large Language Models (LLMs) such as ChatGPT and Llama have impressed us with what they can do. They have also horrified us with what they actually do when they are employed with no protection: hallucinations, stale knowledge bases, no conceptual basis for reasoning, and a capacity for toxic and inappropriate content generation. Rather than avoid them altogether or risk legal liability or brand damage, we can put some guardrails around them to benefit from their best traits without fearing their worst.

Retrieval Augmented Generation (RAG) systems augment the process to make it behave more to our liking. Come hear what you can do to benefit from AI systems without fearing them.

We will cover examples using LangChain and LlamaIndex, two open source frameworks for working with LLMs and creating RAG infrastructure.

We will cover:

Introduction to LLMs
Risks and Limitations
Basic RAG Systems
Embeddings
Vector Databases
Prompt Engineering
Testing and Validating LLMs and RAG Systems
Advanced Techniques
AI as Judge

Advanced RAG : Multimodal and Agentic Systems

Wednesday, 3:15 PM EST

We have seen how Retrieval Augmented Generation (RAG) systems can help prop up Large Language Models (LLMs) to avoid some of their worst tendencies. But that is just the beginning. The cutting edge stateoftheart systems are Multimodal and Agentic, involving additional models, tools, and reusable agents to break problems down in separate pieces, transform and aggregate the results, and validate the results before returning them to the user.

Come get introduced to some of the latest and greatest techniques for maximizing the value of your LLMbased systems while minimizing the risk.

We will cover:

• The LangChain and LlamaIndex Frameworks
• Naive and Intermediate RAG Systems
• Multimodal Models (Mixing audio, text, images, and videos)
• Chatbots
• Summarization Services
• Agent Protocols
• Agent Design Patterns

Advanced RAG : Multimodal and Agentic Systems

Wednesday, 5:00 PM EST

We have seen how Retrieval Augmented Generation (RAG) systems can help prop up Large Language Models (LLMs) to avoid some of their worst tendencies. But that is just the beginning. The cutting edge stateoftheart systems are Multimodal and Agentic, involving additional models, tools, and reusable agents to break problems down in separate pieces, transform and aggregate the results, and validate the results before returning them to the user.

Come get introduced to some of the latest and greatest techniques for maximizing the value of your LLMbased systems while minimizing the risk.

We will cover:

• The LangChain and LlamaIndex Frameworks
• Naive and Intermediate RAG Systems
• Multimodal Models (Mixing audio, text, images, and videos)
• Chatbots
• Summarization Services
• Agent Protocols
• Agent Design Patterns

Vector Databases : Accelerating Learning and Discovery

Thursday, 9:00 AM EST

If you are getting tired of the appearance of new types of databases… too bad. We are increasingly relying on a variety of data storage and retrieval systems for specific purposes. Data does not have a single shape and indexing strategies that work for one are not necessarily good fits for others. So after hierarchical, relational, object, graph, columnoriented, document, temporal, appendonly, and everything else, get ready for Vector Databases to assist in the systematization of machine learning systems.

This will be an overview of the benefits of vectors databases as well as an introduction to the major players.

We will focus on open source versus commercial players, hosted versus local deployments, and the attempts to add vector search capabilities to existing storage systems.

We will cover:

• A brief overview of vectors
• Why vectors are so important to machine learning and datadriven systems
• Overview of the offerings
• Adding vector search to other systems
• Sample use cases shown with one of the key open source engines

Vector Databases : Accelerating Learning and Discovery

Thursday, 11:00 AM EST

If you are getting tired of the appearance of new types of databases… too bad. We are increasingly relying on a variety of data storage and retrieval systems for specific purposes. Data does not have a single shape and indexing strategies that work for one are not necessarily good fits for others. So after hierarchical, relational, object, graph, columnoriented, document, temporal, appendonly, and everything else, get ready for Vector Databases to assist in the systematization of machine learning systems.

This will be an overview of the benefits of vectors databases as well as an introduction to the major players.

We will focus on open source versus commercial players, hosted versus local deployments, and the attempts to add vector search capabilities to existing storage systems.

We will cover:

• A brief overview of vectors
• Why vectors are so important to machine learning and datadriven systems
• Overview of the offerings
• Adding vector search to other systems
• Sample use cases shown with one of the key open source engines