Decision #006

Stream assistant replies over server-sent events

accepted accepted 2026-03-30 by dana

Full replies take 3โ€“8 seconds; dispatchers during an incident abandon anything that feels frozen. We needed incremental delivery through the existing nginx ingress, which our platform team won't reconfigure for WebSocket upgrade semantics.

sendMessage streams the reply as server-sent events over plain HTTP. The ingress needs only a raised proxy-read-timeout; the console renders chunks as they arrive.

First token on screen in under a second. SSE is one-directional, which is all a reply needs โ€” interrupting a stream is a client-side abort. The heartbeat comment every 15s keeps the ingress from reaping quiet streams.