Use qwen3.8-27b as the example executor
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m22s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m22s
It led the executor benchmark (PR #55): 37/40 over eight tasks and 17/20 on the harder four, ahead of claude-haiku-4-5's 15/20. local-fast, the previous example, scored 0/8 on the harder four. Model selection is on the dev executor alone for now; the shared-host question (spec Part H) stays open. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
dfa08d4900
commit
fe73112aa2
2 files changed
+4
-2
No files matched your search
@@ -23,7 +23,7 @@ llm: # A5: LiteLLM is the only LLM destination
|
|||||||
request_timeout_s: 600
|
request_timeout_s: 600
|
||||||
|
|
||||||
roles: # A11: role -> LiteLLM model alias
|
roles: # A11: role -> LiteLLM model alias
|
||||||
executor: local-fast # the local tier; the benchmark picks the model
|
executor: qwen3.8-27b # the local tier: best executor in the benchmark (PR #55)
|
||||||
planner: claude-opus-5-5 # high-end judgement: plans a weaker executor follows
|
planner: claude-opus-5-5 # high-end judgement: plans a weaker executor follows
|
||||||
delegate: claude-opus-5-5 # takes over what the executor got stuck on
|
delegate: claude-opus-5-5 # takes over what the executor got stuck on
|
||||||
consultant: claude-haiku-4-5 # one question at a time, no tools: cheap is enough
|
consultant: claude-haiku-4-5 # one question at a time, no tools: cheap is enough
|
||||||
|
|||||||
@@ -272,7 +272,9 @@ def test_the_example_file_is_valid() -> None:
|
|||||||
"NISSE_MATRIX_TOKEN": "syt_x",
|
"NISSE_MATRIX_TOKEN": "syt_x",
|
||||||
"NISSE_GITEA_TOKEN": "gt",
|
"NISSE_GITEA_TOKEN": "gt",
|
||||||
}
|
}
|
||||||
assert AgentConfig.from_file(example, env).tuning.executor.max_turns == 40
|
config = AgentConfig.from_file(example, env)
|
||||||
|
assert config.tuning.executor.max_turns == 40
|
||||||
|
assert config.roles.executor == "qwen3.8-27b"
|
||||||
|
|
||||||
|
|
||||||
# --- nothing that carries secrets may sit inside the clone -----------------
|
# --- nothing that carries secrets may sit inside the clone -----------------
|
||||||
|
|||||||
Reference in new issue
Block a user