Stage 3 · Expert · A3
Shipping and Operating AI
The model is ten percent of the system.
12 lessons · 116 minDemanding
About this chapter
The day a model goes live your problems change. The wait people feel is the slowest wait, not the average one. The bill grows with every happy user. Quality slides while the dashboard stays green, because the world moved and the model did not. This chapter is the engineering around the model: serving, cost, watching, drift, feedback, guardrails, releases you can undo, and the notes you promised to delete. It ends with the question nobody asks often enough, which is whether to use a model at all.
What you will be able to do
- 18 min
The Model Is Ten Percent
Name the parts around a model and say which of them your year goes to.
- 210 min
Two Phases and a Cache
Name the two phases of an answer and work out how much conversation one card can hold.
- 311 min
Batching and the Tail
Trade batch size against the wait, and read a system by its slowest users.
- 410 min
The Cost of One Answer
Price a month of real users, find the few who are expensive, and know when renting a card pays.
- 510 min
Watching Quality, Not Uptime
Turn quality into a daily number with a range, and catch a drop in one day.
- 610 min
When the World Moves
Spot drift in what arrives before the outcome tells you, and score how far it moved.
- 710 min
Loops That Help, Loops That Rot
Build a loop that gets better from real failures without feeding on itself.
- 810 min
Guardrails and Fallbacks
Set a guardrail threshold on purpose and build a ladder to catch what it blocks.
- 910 min
Shipping a Change
Decide the size of an experiment before you look, and read two arms honestly.
- 109 min
Versions You Can Roll Back
Pin every version, log it on each row, and undo a bad one in a minute.
- 119 min
What You Keep
Choose a retention window you can defend and make a deletion actually delete.
- 129 min
When Not to Use AI
Ask the three questions that decide whether a model belongs in a decision at all.