Rohit Bhardwaj

AI-Era System Design & Architecture Mastery Katas

Monday, 9:00 AM EST

Classic system design teaches you how to scale requests. AI-era architecture teaches you how to scale reasoning, retrieval, tokens, tools, trust, and cost.

In the AI era, the best architects do not just draw boxes. They design authority, evidence, fallback, observability, and cost controls into every system.

Modern system design has entered a new era. It’s no longer enough to optimize for uptime and latency — today’s systems must also be AI-ready, token-efficient, trustworthy, and resilient. Whether building global-scale apps, powering recommendation engines, or integrating GenAI agents, architects need new skills and playbooks to design for scale, speed, and reliability.

This full-day workshop blends classic distributed systems knowledge with AI-native thinking. Through case studies, frameworks, and hands-on design sessions, you’ll learn to design systems that balance performance, cost, resilience, and truthfulness — and walk away with reusable templates you can apply to interviews and real-world architectures.

Learning Outcomes

By the end of this workshop, participants will be able to:

1. Apply a 7-step AI-era system design framework to both interviews and real-world architecture reviews.
2. Design systems that scale across requests, users, regions, tokens, retrieval calls, model latency, and inference cost.
3. Architect production-grade RAG 2.0 pipelines using chunking, embeddings, hybrid search, reranking, GraphRAG, semantic caching, and freshness controls.
4. Design agentic architectures with bounded tools, permission scopes, human escalation, audit trails, and safe degradation.
5. Build multi-provider resilience strategies using model routing, fallback models, circuit breakers, budget caps, and graceful degradation ladders.
6. Apply AI security patterns for prompt injection, indirect prompt injection, data exfiltration, unsafe tool use, and authorization bypass.
7. Define AI observability and SLOs for hallucination rate, grounding quality, retrieval precision, drift, latency, token cost, and escalation rate.
8. Practice architecture katas for e-commerce, ride-sharing, search, fraud detection, social feed, video streaming, and AI assistants.
9. Defend trade-offs under interview pressure using clear diagrams, capacity estimates, failure-mode reasoning, and cost-aware decisions.
10. Leave with reusable templates for AI System Design Canvas, RAG Checklist, Agent Safety Checklist, Token Capacity Planner, AI SLO Dashboard, and Chaos Runbook.

AI Inference at Scale: Reliability, Observability, Cost & Sustainability

Tuesday, 8:30 AM EST

AI inference is no longer a simple model call—it is a multi-hop DAG of planners, retrievers, vector searches, large models, tools, and agent loops. With this complexity comes new failure modes: tail-latency blowups, silent retry storms, vector store cold partitions, GPU queue saturation, exponential cost curves, and unmeasured carbon impact.

In this talk, we unveil ROCS-Loop, a practical architecture designed to close the four critical loops of enterprise AI:
•Reliability (Predictable latency, controlled queues, resilient routing)
•Observability (Full DAG tracing, prompt spans, vector metrics, GPU queue depth)
•Cost-Awareness (Token budgets, model tiering, cost attribution, spot/preemptible strategies)
•Sustainability (SCI metrics, carbon-aware routing, efficient hardware, eliminating unnecessary work)

KEY TAKEAWAYS
•Understand the four forces behind AI outages (latency, visibility, cost, carbon).
•Learn the ROCS-Loop framework for enterprise-grade AI reliability.
•Apply 19 practical patterns to reduce P99, prevent retry storms, and control GPU spend.
•Gain a clear view of vector store + agent observability and GPU queue metrics.
•Learn how ROCS-Loop maps to GCP, Azure, Databricks, FinOps & SCI.
•Leave with a 30-day action plan to stabilize your AI workloads.

⸻

AGENDA
1.The Quiet Outage: Why AI inference fails
2.X-Ray of the inference pipeline (RAG, agents, vector, GPUs)
3.Introducing the ROCS-Loop framework
4.19 patterns for Reliability, Observability, FinOps & GreenOps
5.Cross-cloud mapping (GCP, Azure, Databricks)
6.Hands-on: Diagnose an outage with ROCS
7.Your 30-day ROCS stabilization plan
8.Closing: Becoming a ROCS AI Architect

Architecting Microservices for Agentic AI Integration

Tuesday, 10:30 AM EST

Autonomous LLM agents don’t just call APIs — they plan, retry, chain, and orchestrate across multiple services.
That fundamentally changes how we architect microservices, define boundaries, and operate distributed systems.
This session delivers a practical architecture playbook for Agentic AI integration — showing how to evolve from simple request/response designs to resilient, event-driven systems.
You’ll learn how to handle retry storms, contain failures with circuit breakers and bulkheads, implement sagas and outbox patterns for correctness, and version APIs safely for long-lived agents.
You’ll leave with reference patterns, guardrails, and operational KPIs to integrate agents confidently—without breaking production systems.

Problems Solved

• Microservices collapse under agent retries or fan-out behavior
• Lack of event logs or compensations breaks agent re-planning
• Failures cascade due to missing bulkheads or circuit breakers
• Non-deterministic APIs cause unpredictable agent actions
• Ops teams can’t separate or monitor agent vs human traffic

Why Now

• Agentic frameworks (Agentforce, LangGraph, CrewAI) are entering production.
• Traditional microservices assume human or synchronous clients — not autonomous retriers.
• Reliability, determinism, and observability must now be built into API contracts.
• Agent traffic adds new stress patterns and compliance visibility requirements.

What Is Agentic AI in Microservices

• Agents plan, retry, and chain service calls — requiring deterministic, idempotent APIs.
• Services must be tool-callable (stable operationId, strict input/output schemas).
• Systems must survive retry storms, fan-out, and long-lived sessions.

Agenda
Opening: The Shift to Agent-Driven Systems
How autonomous agents change microservice assumptions.
Why request/response architectures fail when faced with planning, chaining, and self-healing agents.

Pattern 1: Event-Driven Flows
Use events, queues, and replay-safe designs to decouple agents from synchronous APIs.
Patterns: pub/sub, event sourcing, and replay-idempotency.

Pattern 2: Saga and Outbox Patterns
Manage long workflows with compensations.
Ensure atomicity and reliability between DB and event bus.
Outbox → reliable publish; Saga → rollback on failure.

Pattern 3: Circuit Breakers and Bulkheads
Contain agent-triggered failure storms.
Apply timeout, retry, and fallback policies per domain.
Prevent blast-radius amplification across services.

Pattern 4: Service Boundary Design
Shape services around tasks and domains — not low-level entities.
Example: ReserveInventory, ScheduleAppointment, SubmitClaim.
Responses must return reason codes + next actions for agent clarity.
Avoid polymorphic or shape-shifting payloads.

Pattern 5: Integrating Agent Frameworks
Connect LLM frameworks (Agentforce, LangGraph) safely to services.
Use operationId as the agent tool name; enforce strict schemas.
Supervisor/planner checks between steps.
Asynchronous jobs: job IDs, progress endpoints, webhooks.

Pattern 6: Infrastructure and Operations

• Observability: Tag agent runs (x-agent-run-id), trace retries, success/failure.
• Versioning: Use SemVer, deprecation headers, and multi-version gateways.
• Resilience: Autoscale on retry rate, degrade gracefully, and run failover drills.

Wrap-Up: KPIs and Guardrails for Production
Key metrics: retry rate, success ratio, agent throughput, event replay lag.
Lifecycle governance: monitoring, versioning, deprecation, and sunset plans.

Key Framework References

• Salesforce Agentforce – agentic orchestration and guardrail templates
• LangGraph / CrewAI – multi-agent planning and coordination patterns
• Cloud Native Patterns: Saga, Outbox, Circuit Breaker, Bulkhead, Event-Driven Architecture
• OpenTelemetry + Prometheus: Observability for agent vs human traffic
• OWASP LLM Top-10: Guardrails for safe function calling and data handling

Takeaways

• Blueprint for agent-friendly microservices architecture
• Patterns for event-driven, saga, and outbox consistency
• Guardrails: circuit breakers, bulkheads, least privilege APIs
• Framework integration checklist (Agentforce, LangGraph, etc.)
• Ops playbook for observability, versioning, and resilience
• KPIs to measure readiness: retry rate, grounding accuracy, and agent success ratio

Beyond APIs: Orchestration Patterns & MCP for Multi-Agent Systems

Tuesday, 1:00 PM EST

Enterprises are moving from single AI agents to networks of agents that trigger thousands of API calls, retries, and tool-chains per prompt. Without orchestration discipline and APIs built for AI-scale, systems buckle under bursty load, retry storms, cache-miss spikes, inconsistent decisions, and runaway costs.

This talk shows how to combine MCP (Model Context Protocol) with proven inter-agent orchestration patterns — Supervisor, Pub/Sub, Blackboard, Capability Router — and how to harden APIs for autonomous traffic using rate limits, dedupe, backpressure, async workflows, resilient caching, and autoscaling without bill shock.

You’ll also learn the AIRLOCK Framework for governing multi-agent behavior with access boundaries, identity checks, rate controls, least-privilege routing, observability, compliance filters, and kill-switches.

You will walk away with a practical blueprint for building multi-agent systems that are fast, safe, reliable, and cost-predictable.

KEY TAKEAWAYS
Pattern Literacy: When to use Orchestrator, Pub/Sub, Blackboard, Router

MCP Fluency: Standardize agent↔tool integration

API Scaling: Rate limits, dedupe, backpressure, async, caching

Resilience: Bulkheads, jitter, circuit breakers, autoscaling guardrails

Observability: Trace chain-ID/tool-ID across agents & tools

AIRLOCK Governance: Access boundaries, identity, rate controls, least-privilege routing, compliance, kill-switches

AGENDA

• Why AI Changes Load Patterns
  Bursty workloads · fan-out · retry amplification · cost spikes

• MCP 101
  Standardized agent→tool access · hot-swappable tools

• Orchestration Patterns
  Supervisor · Pub/Sub · Blackboard · Capability Router

• Architecting APIs for AI Traffic
  Multi-dimensional rate limits · dedupe · backpressure · SWR caching · async

• Resilience & Autoscaling
  Circuit breakers · bulkheads · kill-switches · budget caps

• Observability & Governance
  Chain-ID tracing · anomaly detection · AIRLOCK boundaries

Run an AWS Generative-AI Well-Architected Review

Tuesday, 3:00 PM EST

A live, end-to-end walkthrough of an AWS Well-Architected Review for a GenAI app. You’ll learn how to apply the AWS Generative AI Lens across the six pillars, then add Bedrock Guardrails and Knowledge Bases (RAG) to raise reliability, safety, and accuracy. You’ll leave with a reusable checklist and a prioritized remediation plan.

Who it’s for & why

• Cloud architects, AI/ML engineers, SRE/Platform teams, and security leads
• Moving from PoCs → production with need for a repeatable WA review process

What you’ll learn

• How to apply Generative AI Lens questions per pillar
• Spot hot spots: data flow, prompts, vector DBs, GPU scaling
• Map risks → fixes with Guardrails + RAG patterns

What you’ll take away

• WA Review scorecard & template
• Pillar-by-pillar remediation backlog
• Sample Guardrail policies (safety, PII, toxic output)
• RAG/Knowledge Base reference architecture

Claude Certified Architect Readiness for Developers

Tuesday, 5:00 PM EST

Certification-readiness talk with architecture scenarios, exam-domain mapping, practical examples, and production-design guidance.
Claude is no longer just a chatbot for writing answers. It is becoming part of how developers design, build, review, and automate software. Claude Code can help developers work across repositories, Claude Code GitHub Actions can respond to issues and pull requests, MCP can connect Claude to external tools and systems, and the Claude Agent SDK enables developers to build custom agentic workflows. This creates a new skill requirement for architects: knowing how to design Claude-powered systems that are safe, measurable, governable, and production-ready.
This talk provides a practical readiness roadmap for developers and architects preparing for Claude architecture work and Claude certification-style expectations. We will cover Claude platform fundamentals, Claude Code workflows, MCP/tool governance, Agent SDK patterns, API design, RAG, evals, observability, security, and enterprise deployment concerns. Participants will also work through certification-style scenarios that test architectural judgment, not memorization.

The goal is simple: do not just learn Claude. Learn how to architect with Claude.
Claude's certification should not be treated as a badge. It should be treated as proof that an architect can design safe, production-ready Claude-powered systems.
Main audience promise
By the end of the talk, participants will understand what they need to study, practice, and demonstrate to become Claude architecture-ready.
They will leave with:

1. A Claude architecture domain map
2. A certification-readiness roadmap
3. A practical Claude Code workflow model
4. An MCP/tool governance model
5. Production architecture patterns for Claude-powered systems
6. Scenario-style questions for self-assessment
7. A 30-day preparation plan
   Primary focus
   This is not a generic Claude intro. It is a Claude architecture readiness talk.
   It should cover:
8. Claude platform landscape
9. Claude Code workflow
10. Agent SDK
11. MCP/tool use
12. Claude API patterns
13. Claude-powered RAG
14. agentic workflows
15. context management
16. prompt and instruction design
17. evals
18. observability
19. security
20. data boundaries
21. cost awareness
22. deployment and governance
23. certification-style scenarios

GraphRAG & Explainable AI: Building Trustworthy LLM Outputs

Wednesday, 9:00 AM EST

Most enterprise LLM failures aren’t technical — they’re trust failures. Models hallucinate, drift from source truth, or produce outputs with no provenance. For regulated industries, that’s unacceptable.
This session introduces GraphRAG — a breakthrough approach combining knowledge graphs (Neo4j) with retrieval-augmented generation to deliver traceable, explainable, and auditable AI outputs.
You’ll learn how to design, evaluate, and deploy GraphRAG architectures aligned with the EU AI Act, NIST AI Risk Management Framework, and enterprise AI governance standards.

Problems Solved

• LLM answers without evidence or traceability
• Stale or inconsistent retrieval data
• Non-compliance with transparency and provenance regulations
• Lack of explainability for model outputs
• Low confidence from regulators, auditors, and executives

Why Now

• Enterprise AI adoption slowed by lack of trust and explainability
• Regulations (EU AI Act, NIST AI RMF) now require provenance and model transparency
• Executives demand evidence-based reasoning, not black-box answers

What GraphRAG Is

• Combines knowledge graphs (Neo4j) with retrieval-augmented generation
• Returns answers with structured evidence paths — connecting entities → relationships → source documents → LLM response
• Goes beyond flat vector search to capture contextual meaning, hierarchy, and causality

Where It Applies

• Insurance: Claims approvals and denials with transparent justification
• Healthcare: Patient summaries with provenance and compliance
• Finance: Audit trails, credit-risk reasoning, regulatory reporting
• Policy & Legal: Regulatory interpretation and case law summaries

Why It’s Valuable

• Establishes trust with executives, auditors, and regulators
• Improves faithfulness, groundedness, and transparency of model outputs
• Reduces disputes, compliance risks, and hallucination-related rework
• Creates structured AI reasoning pipelines aligned with governance frameworks

Agenda
Opening & Problem Context
Why trust is the bottleneck for enterprise AI.
Examples of LLMs failing in regulated use cases — what breaks when outputs lack provenance.
Pattern 1: Anatomy of GraphRAG
Understanding how GraphRAG extends RAG with Neo4j graphs.
Schema design for entities, relationships, and evidence paths.
Structured retrieval from graph → vector → generator.

Pattern 2: Architecture & Data Flow
End-to-end GraphRAG blueprint:
Ingestion → Entity extraction → Graph population → Retrieval orchestration → Response grounding.
Contrast with plain RAG and vector-only approaches.

Pattern 3: Explainability & Evaluation
Metrics for evaluating explainability:
Faithfulness, groundedness, and coverage.
How to trace model answers back to graph nodes and documents.
Integration with AI observability platforms (PromptLayer, Arize, etc.).

Pattern 4: Compliance & Governance Alignment
Connecting GraphRAG design to regulatory frameworks:

• EU AI Act: Transparency, traceability, human oversight
• NIST AI RMF: Trustworthiness and accountability
• ISO 42001: AI Management Systems
Implementing provenance tags and explainability layers as compliance enablers.

Pattern 5: Real-World Scenarios
Industry case patterns:

• “Why was this insurance claim denied?”
• “Which regulation does this contract violate?”
• “Which patient data contributed to this summary?”
Each example maps relationships, evidence, and trace paths through Neo4j.

Wrap-Up & Discussion
Recap of GraphRAG architecture and design patterns.
Checklist for adoption: schema templates, metrics, and governance integration.
Q/A and enterprise discussion on explainable AI roadmaps.

Key Framework References

• Microsoft GraphRAG: Open-source structured hierarchical retrieval pattern
• Neo4j Graph Data Science & LLM Integration Guide
• EU AI Act & NIST AI RMF: Provenance, explainability, and risk transparency
• ISO/IEC 42001: AI governance and management principles
• Gartner & Forrester: Trust and transparency as core adoption barriers

Takeaways

• GraphRAG design blueprint (schema + ingestion + retriever)
• Evaluation metrics: faithfulness, groundedness, coverage
• Reference architecture diagrams for Neo4j + RAG + LLM stack
• Playbook for integrating explainability with compliance frameworks

From Brute Force to Brilliance: Algorithmic Thinking in the Age of AI

Wednesday, 11:00 AM EST

Coding interviews and production systems share the same challenge: transforming vague problems into correct, efficient, and explainable solutions.
This talk introduces a 7-step algorithmic thinking framework that begins with a brute-force baseline and evolves toward an optimized, production-grade solution—using AI assistants like ChatGPT and GitHub Copilot to accelerate ideation, edge-case discovery, and documentation, without sacrificing rigor.
Whether you’re solving array or graph problems, optimizing data pipelines, or refactoring legacy logic, this framework builds the discipline of clarity before optimization—and shows how to use AI responsibly as a thinking partner, not a shortcut.

Why This Talk Now (in the AI Era)

• AI is already in your workflow: 51% of professional developers use AI tools daily; 84% plan to adopt. (Stack Overflow Developer Survey)
• AI boosts productivity, but needs structure: Controlled studies show developers complete tasks ~56% faster with GitHub Copilot—but correctness still requires disciplined reasoning. (arXiv)
• Engineering leaders demand ROI + rigor: 71% of organizations report regular GenAI use, but need trustworthy frameworks to reduce “hallucination debt.” (McKinsey)
• Interviews still test DS&A: Problem-solving frameworks outperform memorization. (Google Tech Dev Guide)

Problems Solved

• Unclear or incomplete problem statements
• Over-reliance on AI code suggestions without validation
• Jumping to optimization before correctness
• Failing to reason about time/space complexity
• Difficulty communicating trade-offs in reviews or interviews

The 7-Step Algorithmic Thinking Playbook

1. Clarify – Define inputs, outputs, and constraints precisely.

2. Baseline – Write the simplest brute-force solution for correctness.

3. Measure – Analyze time and space complexity; identify bottlenecks.

4. Map Patterns – Recognize the family (array, tree, graph, DP, greedy).

5. Refactor – Apply the optimal pattern or data structure.

6. Validate – Test edge cases and boundary conditions automatically.

7. Explain – Communicate trade-offs, scalability, and readability.

Learning Outcomes

• Apply a repeatable, 7-step problem-solving framework for any coding challenge.
• Know when brute force is acceptable—and when optimization matters.
• Confidently compare greedy vs. DP or iterative vs. recursive strategies.
• Use AI tools responsibly for ideation, validation, and refactoring.
• Communicate algorithmic reasoning clearly in code reviews and interviews.

Agenda
Opening: The AI-Accelerated Engineer
How AI is reshaping developer workflows—and why algorithmic clarity matters more than ever.
Examples of AI code that’s correct syntactically but wrong logically.

Pattern 1: Clarify and Baseline
Turning vague questions into crisp specifications.
Why starting with brute force improves correctness and confidence.

Pattern 2: Measure and Map Patterns
How to quickly estimate complexity and identify known solution families.
Mapping problems to arrays, graphs, or DP templates.

Pattern 3: Refactor with AI as a Partner
Using Copilot or ChatGPT to suggest refactors, not replace reasoning.
Prompt patterns for safe collaboration (“generate + verify + explain”).
Spotting hallucinated optimizations.

Pattern 4: Validate and Explain
Building automated test scaffolds and benchmark harnesses.
AI-assisted edge-case discovery.
How to articulate trade-offs in interviews or design docs.

Pattern 5: Framework in Action
Live problem walkthrough:
From brute-force substring search → optimized sliding window solution → complexity and trade-off explanation.
Demonstrate where AI adds value and where human logic rules.

Pattern 6: Guardrails for AI-Assisted Coding
Version control hygiene, reproducibility, test coverage.
Ensuring deterministic, reviewable AI suggestions.
Avoiding “hallucination debt” in production codebases.

Wrap-Up: From Algorithms to Systems Thinking
How this framework extends from whiteboard problems to microservices, pipelines, and data workflows.
Checklist for using AI as a disciplined amplifier of human reasoning.

Key Framework References

• Stack Overflow Developer Survey (2024) – AI adoption statistics
• GitHub Copilot Research – Productivity vs correctness studies
• McKinsey State of AI Report – ROI benchmarks in engineering teams
• Google Tech Dev Guide – Problem-solving and DS&A frameworks
• IEEE/ACM Ethical AI Practices – Human-in-the-loop coding

Takeaways

• 7-Step Algorithmic Thinking Framework — printable reference card
• AI Guardrails Checklist for safe Copilot/ChatGPT use in code and reviews
• Prompt Templates for structured ideation, verification, and documentation
• Live Case Study Walkthrough for clarity, optimization, and explanation
• A mindset shift: from memorizing algorithms → to designing reasoning systems

Claude Code Architecture: From Vibe Coding to Governed Engineering Systems

Wednesday, 3:15 PM EST

Claude Code is not just a coding assistant. Used casually, it can create fast prototypes. Used architecturally, it can become a powerful engineering accelerator for discovery, refactoring, test generation, documentation, architecture reviews, and modernization.

This talk teaches architects, tech leads, and senior developers how to use Claude Code as part of a governed software delivery system. We will explore how to structure repositories, write effective CLAUDE.md guidance, create architecture guardrails, generate tests, review AI-produced code, and use Claude Code without turning your codebase into an ungoverned “vibe coding” experiment.

The core message is simple: Claude Code should not replace architecture judgment. It should amplify it.

Anthropic’s own Claude documentation emphasizes prompting clarity, examples, structured guidance, and agentic workflows, which makes architecture-level instructions especially important when using Claude in engineering systems.

Learning Outcomes

Participants will learn how to:

1. Use Claude Code as an architecture assistant, not just a code generator.
2. Create a strong CLAUDE.md file for repo-specific engineering guidance.
3. Apply architecture guardrails for SOLID, API design, testing, security, and maintainability.
4. Use Claude Code for refactoring, modernization, test generation, and documentation.
5. Review AI-generated code using architecture quality gates.
6. Avoid common risks: over-generation, hidden coupling, insecure code, weak tests, and design drift.

Agenda

1. Why Claude Code Changes Software Architecture
   From pair programmer to repo-aware engineering agent.
2. Claude Code Is Not Your Architect
   What Claude can automate, and what humans must still decide.
3. The Architecture Control File: CLAUDE.md
   Repo context, coding standards, architecture rules, test expectations, and anti-patterns.
4. Architecture Katas with Claude Code
   Refactor a service, add tests, improve API boundaries, document architecture decisions.
5. Quality Gates for AI-Generated Code
   SOLID, security, performance, observability, testability, and maintainability checks.
6. Modernization and Refactoring Workflows
   Using Claude Code for legacy analysis, decomposition, migration planning, and safe incremental change.
7. Security, Governance, and Team Adoption
   Permissions, secrets, prompt injection, review workflows, and responsible usage.
8. Final Playbook
   How to introduce Claude Code into an engineering team without losing architecture discipline.

From APIs to MCPs: Making Microservices Agent-Ready with Semantic Context

Wednesday, 5:00 PM EST

LLM agents don’t just fetch data—they decide and act. To support planning and chaining, microservices must expose not only endpoints but also semantic context: what entities mean, which states are valid, which actions come next, and why decisions were made. This talk shows how to evolve from data-only APIs to MCP-aware, semantically rich services using JSON-LD/Schema.org, Hydra-style affordances, domain events, and OpenAPI metadata. You’ll learn retrofit vs greenfield paths, see cross-industry demos, and leave with a migration checklist that makes your services truly agent-ready.

Agenda

• Why MCPs matter: the “USB-C for AI apps.”
• Gaps in data-only APIs: no semantics, state, or lineage.
• Adding semantics: JSON-LD, Hydra, enriched OpenAPI, domain events.
• How agents benefit: safer chaining, fewer errors, explainability.
• Implementation paths: retrofit vs MCP-native.
• Use cases & demos: retail discounts, manufacturing sensors, insurance risk, healthcare consent.

Takeaways

• MCPs transform APIs into agent-collaborators, not just data pipes.
• Semantic context = better planning, safer automation, fewer hallucinations.
• Practical roadmap: enrich existing APIs or design new ones MCP-native.

Securing LLMs in Production: From OWASP Top-10 to Guardrails that Work

Thursday, 9:00 AM EST

Large Language Models unlock new capabilities—and expose brand-new attack surfaces.
From prompt injection and data exfiltration to model denial-of-service and insecure plugin calls, adversaries are exploiting weaknesses traditional AppSec never anticipated.
The new OWASP LLM Top-10 provides a shared vocabulary for AI risks; this session turns that list into actionable engineering practice.
You’ll learn how to threat-model LLM endpoints, design guardrails that actually block malicious behavior, sandbox tools and plug-ins with least privilege, and align your mitigations to the NIST AI Risk Management Framework for audit-ready governance.

Problems Solved

• Unprotected LLM endpoints vulnerable to prompt injection or jailbreaks
• Lack of policy filters or content-moderation layers
• Plugins/tools running with excessive privileges
• Poor data isolation leading to tenant cross-leakage in RAG systems
• Missing visibility into misuse, drift, and attack attempts

Why Now

• OWASP LLM Top-10 (2024–2025) defines a standard taxonomy of AI risks—widely adopted across the industry.
• AI platforms are moving to production faster than security controls can catch up.
• Regulators and auditors require demonstrable alignment with frameworks like NIST AI RMF and ISO 42001.
• Attackers have begun weaponizing generative models through indirect prompt injection and model-driven DoS.

What You’ll Learn

• Threat-modeling methodology for LLM endpoints and agentic flows
• Input/output guardrail design: policy filters, allow-lists, block-lists, and content classifiers
• Sandboxing tools, plug-ins, and function calls with least privilege and egress control
• Sensitive-data redaction and tenancy-aware retrieval for secure RAG pipelines
• Red-team drills mapped to OWASP categories and mitigation validation
• How to map each control to NIST AI RMF (governance, risk, assurance)

Agenda
Opening: The New AI Attack Surface
How LLMs change the threat model. Examples of real-world attacks: prompt injections, indirect injections, model DoS, and exfiltration via vector stores.

Pattern 1: Threat Modeling LLM Endpoints
Identify assets, trust boundaries, and high-risk flows.
Apply STRIDE-inspired analysis to prompts, context windows, retrieval layers, and plugin calls.

Pattern 2: Designing Input/Output Guardrails
Policy filtering, schema validation, and content moderation.
Runtime vs compile-time guardrails—what actually works in production.
Enforcing determinism and fail-safe defaults.

Pattern 3: Sandboxing and Least Privilege Plugins
Secure function calling: scoped IAM, network egress rules, per-plugin secrets, and API key vaulting.
Container isolation and ephemeral agent sandboxes.

Pattern 4: Data Protection and Tenancy in RAG
Redacting sensitive data before embedding.
Segregating tenant vectors and access policies.
Auditing data lineage and evidence paths.

Pattern 5: Red Team & Evaluation Frameworks
Running adversarial simulations aligned with OWASP LLM Top-10.
Common exploits and how to detect them.
Integrating automated red-team tests into CI/CD pipelines.

Pattern 6: Governance & Framework Mapping
Mapping mitigations to NIST AI RMF (categories RA, MA, ME).
Building dashboards and executive summaries for risk reporting.

Wrap-Up & Action Plan
Summarize practical controls that can be implemented within 30 days.
Introduce the Guardrail Policy Starter Kit + Red-Team Runbook templates.
Live checklist review for readiness maturity.

Key Framework References

• OWASP LLM Top-10 (2024–2025) – Prompt Injection, Data Exfiltration, DoS, Insecure Plugins
• NIST AI RMF (2023) – Governance + Risk + Assurance Categories
• ISO/IEC 42001 – AI Management System Standard
• MITRE ATLAS – Adversarial Tactics for AI Systems
• OWASP SAMM + ASVS – Integrating AI security into AppSec programs

Takeaways

• Clear understanding of OWASP LLM Top-10 risks in plain language
• Guardrail Policy Starter Kit (template YAML + reference policies)
• Sandboxing Playbook for tools and plugins (scoped IAM, network controls)
• Red-Team Runbook for testing and validation
• NIST AI RMF Mapping Guide for executive and audit reporting
• A practical 30-day roadmap to move from reactive patching → resilient AI security