Keep runs with different tuning apart in the report, and raise max_turns to 40
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m11s

The report grouped runs by model, serving note and task version, so
runs made at different tuning were silently averaged together. Tuning
is now part of the key, and a tuning column shows how a row's settings
differ from today's defaults; it appears only when some row needs it.

tuning.executor.max_turns goes from 24 to 40. On the harder benchmark
tasks Haiku, the calibration ceiling, hit 24 on both refactor-rename
runs and both long-milestone runs, one of which had finished the work.
The cap guards against a model trying forever; it isn't a budget.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-09-27 22:25:14 +00:00
1 parent a77c84c128
commit f881a18ca4
6 files changed
+72 -13

No files matched your search

+6 -1
View File
@@ -147,7 +147,7 @@ every request, on both routes.
expect, which is the model's output ceiling divided by its generation
speed.
### `tuning.executor.max_turns` — default 24
### `tuning.executor.max_turns` — default 40
How many model calls one milestone may make before the executor stops with
`capped` and asks you what to do.
@@ -161,6 +161,11 @@ How many model calls one milestone may make before the executor stops with
the cap comfortably above the high end, around 1.5–2× the turns of your
longer successful runs. A stronger model usually needs fewer turns, not
more.
- **Measured:** the default was 24 until the harder benchmark tasks arrived.
Haiku, used as the ceiling in calibration, hit 24 on both runs of
`refactor-rename` (editing ten files one at a time) and of
`long-milestone`, one of which had in fact finished the work. It is a guard
against a model trying forever, not a budget, so it was raised to 40.
### `tuning.executor.failure_limit` — default 2