Pick the host before you stream, and prewarm
Practice
The practice
Streaming cannot retry mid-response. Check /health first, then send a one-token non-streamed request to pay the prefill, because the shared server sometimes aborts a streamed generation whose prompt is not cached.
2026-10-09
Comments
Public comments on each entry are coming. Nothing is collected here yet.