Render one assistant message per turn, and make the request timeout a setting
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m16s

Two setup failures found while calibrating the new benchmark tasks,
both of which would also fail real milestones.

Devstral's chat template (Mistral) rejected any turn in which the model
wrote some text before a tool call: the OpenAI renderer made one message
per event, so the turn became a text message and a tool-call message in
a row, and llama.cpp answered 500. The renderer now folds consecutive
user or assistant messages into one, which is the OpenAI format for a
turn anyway. Checked against the live server: the captured failing
request no longer occurs, over six real runs.

LiteLLMClient gave up on any request after a fixed 120 s. A dense 27B
model writing a whole module in one turn takes longer, so the executor
escalated work that was going well. llm.request_timeout_s now sets it
for both clients, default 600.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-09-27 22:15:30 +00:00
1 parent f41cbb00cf
commit d49435d4e5
8 files changed
+139 -8

No files matched your search

+15
View File
@@ -132,6 +132,21 @@ a local model can be held to less without affecting the frontier roles.
executor. Frontier planning wants the large default. Leave this high, and
set a local model's smaller ceiling as its `max_output_tokens` in LiteLLM.
### `llm.request_timeout_s` — default 600
How long one model request may take before it is abandoned. It applies to
every request, on both routes.
- **Too low:** a slow model's long answer is cut off, and the executor
escalates work that was going well. At 120 s, the old fixed value, a dense
27B model on the RTX 4090 timed out writing one module in one turn.
- **Too high:** a hung connection holds the milestone for that long before
anything notices.
- **Deciding:** this is a safety net, not a budget. `max_output_tokens` and
`max_turns` bound the work; set this above the slowest single answer you
expect, which is the model's output ceiling divided by its generation
speed.
### `tuning.executor.max_turns` — default 24
How many model calls one milestone may make before the executor stops with