Postmortem culture for AI teams: what breaks in production and why
AI systems fail differently than traditional software. Model provider silent updates, prompt drift, cost explosions, tenant-specific hallucinations. Here is the postmortem template we use at Praxvon — and the four incident types we see most often.
TL;DRAI systems fail in ways traditional SRE playbooks don't cover: silent model updates from providers, prompt drift when copy edits break structure, cost explosions from a single user's abuse, tenant-specific hallucinations that only happen with a subset of data. Regular postmortem culture with a template that covers these classes is how you get better fast.
We inherit AI systems from other teams often enough to have a sample set of how they fail. Not "the model was wrong" style failures — production ops failures that cost time, money, or trust.
This post covers the four incident classes we see most often and the postmortem template we use at Praxvon to make sure the same class doesn't fire twice.
Why AI postmortems are different
Traditional SRE incidents fit a pattern: server crashed, latency spiked, database lost consistency. Root cause is usually a code change, config drift, or resource exhaustion. Postmortems in this world are mature — Google's SRE book, blameless culture, five whys, all of it transfers.
AI systems break in additional ways:
- The model changes underneath you. Providers version and
sometimes silently update models.
gpt-4o-2024-05-13gets a patch that changes outputs. Your eval suite catches it (see Evals in Production). Or it doesn't, and you find out from a customer complaint. - Prompts drift. A well-meaning copy edit changes a system prompt from "return JSON" to "return the JSON" and 3% of responses stop parsing.
- Costs explode from a single user. One aggressive user hits your API 10× more than expected and quadruples the monthly bill.
- Hallucinations are tenant-specific. The system passes evals because your golden set doesn't include Tenant X's data shape. It fails silently on their data for a month.
Traditional postmortem templates miss most of these because they assume the failure surface is code + config.
The four incident classes
1. Model provider silent update
Symptom: Quality regression detected via eval or user complaint. Nothing in your codebase changed.
Cause: OpenAI / Anthropic / Google shipped a model update behind the same version alias.
Prevention: Pin to date-stamped model IDs (gpt-4o-2024-08-06,
not gpt-4o). When a provider deprecates the date-stamped version,
run the eval suite against the new version before rolling forward.
Real example: we had a translation eval drop 8 points overnight in
May 2026. Nothing changed on our side. Anthropic had updated
claude-sonnet-4 to a newer snapshot behind the alias. We pinned to
the date-stamped version and got the 8 points back the next hour.
2. Prompt drift from copy edits
Symptom: A specific downstream feature stops working. Usually a parser breaks or a follow-up step gets malformed input.
Cause: Someone edited a prompt for clarity, tone, or brevity. Structure that the parser depended on was subtly changed.
Prevention: Every prompt change PRs against the eval suite. If the prompt is load-bearing structure, add a specific test case that would fail on a structure break.
Real example: a "return JSON" prompt became "please return the
JSON response". Model started wrapping the JSON in markdown code
fences ```json ... ```. Parser broke on 3% of responses. Fix:
add a case that specifically checks raw JSON parsing.
3. Cost explosion from one user or endpoint
Symptom: Weekly bill 2-4× higher than expected. Usage graph shows one user or endpoint dominating.
Cause: Either abuse (one user hammering the endpoint) or a bug that turned an inner loop into an LLM call.
Prevention: Per-user + per-endpoint cost logging + daily alert threshold (see AI Inference Cost Control). Rate limits per user for anything customer-facing.
Real example: an internal batch job accidentally called an LLM inside a loop that ran once per row instead of once per batch. Silent bill inflation for a week before the daily alert fired. We now have per-endpoint burn-rate alerts, not just totals.
4. Tenant-specific hallucination
Symptom: Customer X reports "the AI keeps getting names wrong." Customer Y has no issues. You can't reproduce X's problem in staging.
Cause: Customer X's data has a shape your golden set doesn't cover. Names in a script the model wasn't trained on, industry jargon your prompts don't anticipate, or a corner case in your RAG retrieval that only surfaces with their embedding distribution.
Prevention: Multi-tenant systems need tenant-representative samples in the eval suite. As you onboard customers, add anonymized samples of their data to the golden set.
Real example: Moditra had translation failures for a customer with Arabic-language chat threads. Eval suite was TR/EN heavy, AR under-represented. Added 8 AR-heavy cases; regression detection improved and the customer's problem resolved within a sprint.
The postmortem template we use
Every incident, no matter how small, gets a document. Template:
Incident: [short description]
Detected: [when + how]
Severity: [P0/P1/P2 with impact statement]
Duration: [start → resolved]
Owner: [name]
Timeline:
[UTC times, what happened, who did what]
Impact:
[users affected, requests failed, cost incurred, customer trust]
Root cause:
[the ONE thing that if different would have prevented this]
Contributing factors:
[things that made it worse or delayed detection]
What went well:
[stuff to keep doing]
What went poorly:
[stuff to fix, blameless]
Action items:
[owner + deadline for each]
- [ ] Add eval case that would have caught this
- [ ] Improve alerting so next occurrence is faster to detect
- [ ] Update runbook with the resolution steps
The section that's non-standard for AI postmortems: "Add eval case that would have caught this." Every AI incident produces a new golden set case. Over time your eval suite becomes a comprehensive map of what has broken and how.
Blameless is the whole thing
The AI failure modes above are structural. Nobody is stupid for missing that OpenAI silently updated a model — it's a systemic issue requiring a systemic fix (pin versions). The postmortem culture that makes teams get better fast is the one where nobody feels bad about writing the doc.
If people are afraid to write postmortems, they hide incidents. Hidden incidents recur. That's how AI teams end up 18 months in with the same problems they started with.
What we don't postmortem
One-off transient model errors (API 500, temporary rate limit). These are noise unless they repeat.
Individual bad completions where the user says "the AI was wrong once." Post-hoc rationalization territory. If it's a pattern across users, that's an incident. Single anecdote isn't.
"The AI is bad at X" where X was never in scope. Not an incident; that's a product decision.
Running an AI system without postmortem discipline and want a practical playbook? Contact us — we can walk you through the setup in an afternoon.