Operations runbook

What pages, what to check, and the three incident playbooks.

Operations runbook

Dashboards

• Token spend — per-namespace billing panel; the daily budget line is the one that pages.
• Turn latency — p95 time-to-first-token; SSE means users feel this number, not total latency.
• Retrieval health — ingest worker queue depth and embedding failures.

Playbook: model provider outage

The assistant degrades, parcel tracking does not (that separation is ADR-001). Set the console banner via dispatch-config, escalate open conversations, and wait it out. Do not point DISPATCH_MODEL at an unevaluated model tier during an outage.

Playbook: runaway token spend

Check conversations.tokens_used outliers first — budget enforcement (ADR-004) should make this structurally impossible; if a conversation exceeded its budget, that is a bug, not a tuning problem.

Playbook: bad prompt in production

Roll back through the prompt registry (ADR-007): activate the previous version — one UPDATE, no deploy. Then file the eval-suite gap that let it through.