Roles, capability profiles and help on the new loop (M2.6)
- roles.driver (unset: the planner's model) and roles.classifier (unset: the expert's); the mapping is pinned when an item opens, and the driver and the go-ahead reader run on the item's own (L9). - profiles: in configuration (tools, strengths, weaknesses), used in briefings: the driver's instructions say what its helper and expert are good at, the helper's brief who reads its handback; never a model name. - delegate(task) and ask_expert(question) for the driver. A helper works on a branch off the milestone's; on a handback its work comes back as uncommitted changes, otherwise it is discarded, so harness branches never hold commits only the clone has. ask_expert keeps its budget. - Triggers: failure_limit failed commands in a row (Driver's new "stuck" outcome) or the turn cap, mid-milestone, call a helper in once per milestone; if it fails too, the item waits on the operator. A delegation cut off by a crash is discarded at boot. - Evaluation capture: delegations and ask_expert as before, and the reader's verdicts as classifier pairs. Tests: help tests (merge-back, discard, needs a milestone, the stuck trigger and its once-per-milestone limit, the expert and its budget, pinning, briefings by profile, boot discarding a cut-off delegation), config tests for roles and profiles, and the walk delegating with a scripted helper that sometimes fails, with a delegations_settle property. The walk found helper commits stranded on a merged PR's branch, fixed by returning a helper's work uncommitted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
1 parent
04c352b30c
commit
59fd00ce8c
19 files changed
+890
-38
No files matched your search
@@ -45,6 +45,27 @@ roles: # L9: role -> LiteLLM model alias
|
||||
planner: claude-opus-5-5 # high-end judgement: plans a weaker executor follows
|
||||
helper: claude-opus-5-5 # takes over what the executor got stuck on
|
||||
expert: claude-haiku-4-5 # one question at a time, no tools: cheap is enough
|
||||
# The new loop's roles (loop: new): unset, driver is the planner's model and
|
||||
# classifier the expert's. Each work item pins the mapping when it opens.
|
||||
# driver: claude-opus-5-5 # drives the loop: talks, restates, works
|
||||
# classifier: claude-haiku-4-5 # reads plain-language go-aheads (later: actions, L4)
|
||||
|
||||
# Capability profiles (L9), from benchmarks: what one role is told about
|
||||
# another (never a model's name). Optional; a model without one is briefed as
|
||||
# "no profile recorded".
|
||||
profiles:
|
||||
qwen3.8-27b:
|
||||
tools: reliable (37/40 on the executor benchmark, PR #55)
|
||||
strengths: careful multi-step edits that follow a clear brief
|
||||
weaknesses: speed (minutes per hard task) and open-ended design
|
||||
claude-haiku-4-5:
|
||||
tools: reliable
|
||||
strengths: quick, well-scoped changes and short answers
|
||||
weaknesses: long multi-file work (15/20 on the harder benchmark tasks)
|
||||
claude-opus-5-5:
|
||||
tools: reliable
|
||||
strengths: design, debugging, and work other models got stuck on
|
||||
weaknesses: cost; use it for what needs it
|
||||
|
||||
matrix: # L5
|
||||
homeserver: https://matrix.example
|
||||
|
||||
Reference in new issue
Block a user