Streaming RAG Why Streaming Matters for RAG RAG end-to-end latency is typically 1.5-4s. Non-streaming feels broken; streaming cuts perceived latency by 5-10x because users start reading within 200ms of generation start. Budget target: - First retrieval status: < 100ms (from request) - Retrieval done: < 800ms - First token: < 1000ms (TTFT) - Full answer: < 3500ms Three Streaming Phases Keep the user informed in phase 1-2 so they do not bail on the request. Server-Side: Python Async Generator (FastAPI + SSE) Six event types over the wire: , , , , , . Heartbeats (required for long retrievals beh…