Decision #003

RAG over fine-tuning for domain knowledge

accepted accepted 2026-02-27 by carla

The assistant needs carrier SLAs, depot runbooks, and refund policy โ€” content that changes weekly. Fine-tuning bakes knowledge into a model artifact that is stale by the next policy update and unauditable when an answer is wrong.

Retrieval-augmented generation: sources are ingested, chunked, and embedded into Postgres (knowledge_chunks); every assistant turn retrieves top-k chunks and cites them. No fine-tuned models.

Wrong answers are debuggable โ€” the retrieval trace is stored on the tool turn. Updating knowledge is a re-ingest, not a training run. Retrieval adds ~120 ms per turn, acceptable inside the streaming UX.