Decision #004

pgvector for embeddings, hard token budgets per conversation

accepted accepted 2026-03-12 by bruno

The retrieval index needs a vector store. The platform already runs Postgres (platform ADR-003) and the ops team refuses a second stateful system below ~10M chunks. Separately, early testing produced conversations that quietly burned six-figure token counts through retry loops.

Embeddings live in Postgres via the pgvector extension with an HNSW index — no dedicated vector database. Every conversation carries a hard token budget (default 200k, CONVERSATION_TOKEN_BUDGET); when exhausted the API returns 429 and the conversation must be escalated to a human.

One database to operate, back up, and restore. Budgets turn the worst-case bill per conversation into a constant. If chunk volume outgrows pgvector, the retrieval interface is the seam to swap behind.