Skip to content
AI systemsPro

Turn one LLM call into a dependable service.

Production LLM Systems

Add service promises, traces, traffic controls, cost routing, failure protection, tool permissions, incident response, canaries, and rollback.

What you will be able to do

Leave with capability, not just vocabulary.

Define service-level indicators, objectives, and error budgets

Trace requests and engineer latency under load

Control spend, retries, fallbacks, and tool permissions

Run incidents and release with canary and rollback evidence

Running example

AskHelpwell, a self-contained support workflow followed from one ticket through a controlled production release.

Prerequisites

Basic familiarity with API requests and LLM applications. No prior LDS course or operations background is required.

Curriculum

Every module earns the next one.

Open any module to review its exact sections. Progress and completion follow you through the course.

8 modules · 8h 30m
01
Module 1

The Production Contract: From Demo to Service

BeginnerFree preview

Topics include AskHelpwell workflow, service promises, SLIs and SLOs, and more.

View 5 sections
  1. 1Meet Helpwell and One Support Ticket
  2. 2Why One Working Call Is Not a Service
  3. 3Define What Counts as a Good Request
  4. 4Turn the Promise Into SLI, SLO, and Error Budget
  5. 5Measure the Starting Version
55 min5 sections
Open module
02
Module 2

Trace One Request: Observability That Explains

IntermediatePro

Topics include logs, metrics, and traces, span trees, GenAI telemetry, and more.

View 5 sections
  1. 1Logs, Metrics, and Traces Answer Different Questions
  2. 2The Request Tree: Trace, Span, Parent, Event
  3. 3Instrument the LLM Call
  4. 4Tokens, Cache, Cost, and Streaming Signals
  5. 5Redaction, Cardinality, Sampling, and Telemetry Cost
60 min5 sections
Open module
03
Module 3

Latency Under Load: Queues, Concurrency, and Backpressure

IntermediatePro

Topics include TTFT and total latency, tail percentiles, queueing and concurrency, and more.

View 5 sections
  1. 1TTFT Is Not Total Latency
  2. 2Average Latency Hides the Tail
  3. 3Arrival Rate, Service Time, and Concurrency
  4. 4Backpressure, Queue Bounds, and Load Shedding
  5. 5Capacity Planning With Measured Distributions
65 min5 sections
Open module
04
Module 4

Spend With Intent: Cache, Route, and Service Tier

IntermediatePro

Topics include token unit economics, cost per good outcome, prompt caching, and more.

View 5 sections
  1. 1The Bill Has More Than Two Token Columns
  2. 2Cost Per Successful Outcome, Not Cost Per Call
  3. 3Prompt Cache Economics and Cache-Safe Layout
  4. 4Standard, Priority, Flex, and Batch Workloads
  5. 5Route by Promise: Model, Tier, Fallback, or Human
60 min5 sections
Open module
05
Module 5

Fail Without Falling Over: Deadlines, Retries, and Breakers

AdvancedPro

Topics include deadline propagation, retry classification, backoff and jitter, and more.

View 5 sections
  1. 1One User Deadline, Many Internal Timeouts
  2. 2Retry Only the Right Failure
  3. 3Exponential Backoff, Jitter, and Retry Budgets
  4. 4Idempotency for Side Effects
  5. 5Circuit Breakers, Fallbacks, and Graceful Degradation
70 min5 sections
Open module
06
Module 6

Permission Every Action: Runtime Guardrails and MCP

AdvancedPro

Topics include policy enforcement, identity, audience, scope, and tenant, MCP authorization, and more.

View 5 sections
  1. 1The Model Proposes; the Policy Layer Decides
  2. 2Identity, Audience, Scope, and Tenant
  3. 3MCP Authorization and Least Privilege
  4. 4Argument Validation, Approval, and Side-Effect Classes
  5. 5Secrets, Redaction, Audit Evidence, and Break-Glass Access
65 min5 sections
Open module
07
Module 7

Run the Incident: Detect, Contain, Recover, Learn

AdvancedPro

Topics include symptoms and causes, incident command, containment and rollback, and more.

View 5 sections
  1. 1A Symptom Is Not a Cause
  2. 2Declare, Assign Roles, and Build the Timeline
  3. 3Contain First: Shed, Disable, Fail Over, or Roll Back
  4. 4Verify Recovery With User-Centered Signals
  5. 5Write a Postmortem That Changes the System
65 min5 sections
Open module
08
Module 8

Release With Evidence: Shadow, Canary, Roll Back, Operate

AdvancedPro

Topics include behavior versioning, shadow traffic, canary releases, and more.

View 5 sections
  1. 1Version Everything That Can Change Behavior
  2. 2Shadow Traffic and Dry-Run Side Effects
  3. 3Canary the Change Against Control and Absolute SLOs
  4. 4Automate Hold, Promote, and Roll Back
  5. 5The Production Readiness Review
70 min5 sections
Open module
Who this course is for

Built for people who need to use the skill.

01

AI engineers moving from demos to services

02

Technical leads reviewing production readiness

03

Platform engineers supporting LLM workloads

Start the course

Begin with The Production Contract: From Demo to Service.

The first module establishes the language and example used throughout the rest of the course.

Open Module 1
Production LLM Systems | Let's Data Science