4 Commits

Include renames

Author	SHA1	Message	Date
dongarandClaude Opus 5	2b7651fb37	Add ask_expert and eval capture (A14/A16) — step 7c ... Completes step 7. - expert.py: one stateless call for a question needing no repo access. Structured output is contracted at the prompt level, since vendors differ on JSON enforcement, so the reply is parsed and validated locally with one retry — tolerating a fenced code block, because models wrap JSON habitually, but not missing fields - The per-task budget is counted from the canonical log rather than memory, so it survives a restart and cannot hand a circling executor a fresh allowance. Exhaustion raises rather than answering: a caller out of consults should get on with the work or escalate - Consults are recorded as delegation_started/completed with role consultant. A16 asks for a "consult/delegation eval set", and reusing the composite events gets consult evaluation for free rather than inventing a parallel record - evals.py pairs starts with completions into eval records, with verdicts kept outside the canonical log: the log records what happened, a verdict is a later judgement about it, and judgements get revised. Schema v3 adds the outcomes table, so a v1 store now traverses two migrations — a better exercise of the hook than one - ask_expert is offered to the executor as a tool only when a desk is configured, in addition to the observable escalation triggers and never instead of them (A12). Consulting is not mutating, so a completion claim backed only by consults is still challenged Found live: passing an empty message list and putting the question in the system prompt fails outright — the Messages API requires at least one non-system message. The question now travels as a synthesised user message that is never appended to the log, which keeps the consult stateless while remaining a valid request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-08 19:48:13 +00:00
dongarandClaude Opus 5	886cb80583	Count only mutating tools as evidence of work ... The previous fix challenged a completion claim made before any tool call, but treated every tool as work — so a model that ran one list_dir and then declared the milestone done was believed, and CI caught the same empty milestone again. Reading a file or listing a directory is the model orienting itself. Only write_file and run_command now count, so a completion claim must be backed by something that changed or verified the tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-07 22:53:20 +00:00
dongarandClaude Opus 5	acee8b96f0	Do not believe a completion claim before any work was done ... CI caught the executor recording an empty milestone as a success: the model answered with prose on its first turn, and the loop reads "no tool call" as "the plan is done". It had therefore finished a milestone having touched nothing, and the milestone would have gone on to publish with no changes. A completion claim is now challenged once when no tool has been called yet — the same shape as the planner's nudge — and a model that persists escalates instead of reporting success. After the first tool call the claim is taken at face value, as before. The live milestone test failed roughly one run in five on this; four consecutive runs pass now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-07 22:19:15 +00:00
dongarandClaude Opus 5	0fd0b2e4a5	Add executor tool loop (A12/A15) — step 5b ... - tools.py: four flat tools (read_file, write_file, list_dir, run_command) per A15's reduced manifest. Deliberately unsandboxed: the VM is the trust boundary (B3) and tool use is auto-approved inside an approved unit of work (A4), so path confinement would add failure modes without adding safety. What is enforced is operational — timeout with process-group kill so a backgrounded server cannot outlive its command, output truncation so a build log cannot eat the context window, closed stdin so prompts fail fast, and a scrubbed environment so project code never sees the agent's credentials - executor.py: renders the log, calls the executor role with the reduced manifest, dispatches tool calls, and checkpoints every result (A13). Escalation is observable, never self-assessed (A12): consecutive command failures and a hard turn cap. Failed reads do not count — looking around is not failing - Until delegation exists (step 7), escalation gates to the human with the failures attached - control.py: start milestone now runs the executor; a stopped milestone gates to AWAIT_INPUT Two bugs found by running it live against real models: - A turn-ending gate tool (submit_plan) left a tool_call with no tool_result, which providers reject outright, so every planning turn after the first failed with a 400. The planner now records the gate as the call's outcome, which is also the truthful transcript - The renderer now emits a placeholder for any tool_call lacking a result, so a crash between call and result (A3) cannot poison the transcript permanently Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-07 16:35:39 +00:00