dongarandClaude Opus 5.5
4904fa0b63
A model call that passed request_timeout_s failed the whole run, though the model was still writing. The chat route, which every local model takes, now streams its replies and puts them back together into the response the server would have sent, so the call record and parsing are unchanged. request_timeout_s becomes the longest the model may go silent. Measured on the 4090 through LiteLLM: an abandoned plain request kept the server busy (a tiny request after it waited 51 s); an abandoned stream freed it (0.3 s). A plain retry would have waited behind the abandoned call, so there is none. The passthrough route, hosted Anthropic models only, stays unstreamed: its cost comes only in a header that streaming loses. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk