Decision #002

Claude as the LLM provider

accepted accepted 2026-02-20 by alice

We evaluated three hosted providers against the dispatch eval suite: answer grounding on tracking events, refusal behavior on out-of-scope questions (medical advice for a driver, legal threats from a customer), and streaming latency to first token.

Use Claude (Sonnet tier) through the Messages API for all assistant turns. The model id is configuration (DISPATCH_MODEL), never hard-coded, so tier upgrades are a config rollout.

Self-hosting an open-weights model was rejected for now: the GPU baseline cost exceeded a year of projected API spend at current volume, and we'd own safety tuning ourselves.

One vendor dependency, mitigated by keeping prompts provider-neutral and the provider call behind the assistant package boundary. Token budgets (ADR-004) keep the bill predictable.