Keep runs with different tuning apart in the report, and raise max_turns to 40
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m11s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m11s
The report grouped runs by model, serving note and task version, so runs made at different tuning were silently averaged together. Tuning is now part of the key, and a tuning column shows how a row's settings differ from today's defaults; it appears only when some row needs it. tuning.executor.max_turns goes from 24 to 40. On the harder benchmark tasks Haiku, the calibration ceiling, hit 24 on both refactor-rename runs and both long-milestone runs, one of which had finished the work. The cap guards against a model trying forever; it isn't a budget. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a77c84c128
commit
f881a18ca4
6 files changed
+72
-13
No files matched your search
@@ -47,7 +47,7 @@ gitea: # B2
|
||||
# read at startup, so a change takes effect on the next restart.
|
||||
tuning:
|
||||
executor:
|
||||
max_turns: 24 # model calls per milestone before it stops as capped
|
||||
max_turns: 40 # model calls per milestone before it stops as capped
|
||||
failure_limit: 2 # consecutive failed commands before escalating (A12)
|
||||
expert:
|
||||
consult_budget: 3 # ask_expert calls per milestone (A14)
|
||||
|
||||
Reference in new issue
Block a user