Use qwen3.8-27b as the example executor
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m22s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m22s
It led the executor benchmark (PR #55): 37/40 over eight tasks and 17/20 on the harder four, ahead of claude-haiku-4-5's 15/20. local-fast, the previous example, scored 0/8 on the harder four. Model selection is on the dev executor alone for now; the shared-host question (spec Part H) stays open. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
dfa08d4900
commit
fe73112aa2
2 files changed
+4
-2
No files matched your search
@@ -23,7 +23,7 @@ llm: # A5: LiteLLM is the only LLM destination
|
||||
request_timeout_s: 600
|
||||
|
||||
roles: # A11: role -> LiteLLM model alias
|
||||
executor: local-fast # the local tier; the benchmark picks the model
|
||||
executor: qwen3.8-27b # the local tier: best executor in the benchmark (PR #55)
|
||||
planner: claude-opus-5-5 # high-end judgement: plans a weaker executor follows
|
||||
delegate: claude-opus-5-5 # takes over what the executor got stuck on
|
||||
consultant: claude-haiku-4-5 # one question at a time, no tools: cheap is enough
|
||||
|
||||
@@ -272,7 +272,9 @@ def test_the_example_file_is_valid() -> None:
|
||||
"NISSE_MATRIX_TOKEN": "syt_x",
|
||||
"NISSE_GITEA_TOKEN": "gt",
|
||||
}
|
||||
assert AgentConfig.from_file(example, env).tuning.executor.max_turns == 40
|
||||
config = AgentConfig.from_file(example, env)
|
||||
assert config.tuning.executor.max_turns == 40
|
||||
assert config.roles.executor == "qwen3.8-27b"
|
||||
|
||||
|
||||
# --- nothing that carries secrets may sit inside the clone -----------------
|
||||
|
||||
Reference in new issue
Block a user