The model is ten percent of the system.
A3 · Shipping and Operating AI
The day a model goes live your problems change. The wait people feel is the slowest wait, not the average one. The bill grows with every happy user. Quality slides quietly while the dashboard stays green, because the world moved and the model did not. This chapter is the engineering around the model: serving, cost, watching, drift, feedback, guardrails, releases you can undo, and the notes you promised to delete. It ends with the question nobody asks often enough, which is whether to use a model at all.
- The Model Is Ten Percent
- Two Phases and a Cache
- Batching and the Tail
- The Cost of One Answer
- Watching Quality, Not Uptime
- When the World Moves
- Loops That Help, Loops That Rot
- Guardrails and Fallbacks
- Shipping a Change
- Versions You Can Roll Back
- What You Keep
- When Not to Use AI