Commit Graph
3 Commits
Author SHA1 Message Date
dongarandClaude Opus 5 886cb80583 Count only mutating tools as evidence of work
CI / checks (push) Successful in 38s
CI / checks (pull_request) Successful in 41s
The previous fix challenged a completion claim made before any tool
call, but treated every tool as work — so a model that ran one
list_dir and then declared the milestone done was believed, and CI
caught the same empty milestone again.

Reading a file or listing a directory is the model orienting itself.
Only write_file and run_command now count, so a completion claim must
be backed by something that changed or verified the tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:53:20 +00:00
dongarandClaude Opus 5 acee8b96f0 Do not believe a completion claim before any work was done
CI / checks (push) Successful in 44s
CI / checks (pull_request) Successful in 41s
CI caught the executor recording an empty milestone as a success: the
model answered with prose on its first turn, and the loop reads "no
tool call" as "the plan is done". It had therefore finished a
milestone having touched nothing, and the milestone would have gone
on to publish with no changes.

A completion claim is now challenged once when no tool has been
called yet — the same shape as the planner's nudge — and a model that
persists escalates instead of reporting success. After the first tool
call the claim is taken at face value, as before.

The live milestone test failed roughly one run in five on this; four
consecutive runs pass now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:19:15 +00:00
dongarandClaude Opus 5 0fd0b2e4a5 Add executor tool loop (A12/A15) — step 5b
CI / checks (push) Failing after 13s
CI / checks (pull_request) Failing after 13s
- tools.py: four flat tools (read_file, write_file, list_dir,
  run_command) per A15's reduced manifest. Deliberately unsandboxed:
  the VM is the trust boundary (B3) and tool use is auto-approved
  inside an approved unit of work (A4), so path confinement would add
  failure modes without adding safety. What is enforced is
  operational — timeout with process-group kill so a backgrounded
  server cannot outlive its command, output truncation so a build log
  cannot eat the context window, closed stdin so prompts fail fast,
  and a scrubbed environment so project code never sees the agent's
  credentials
- executor.py: renders the log, calls the executor role with the
  reduced manifest, dispatches tool calls, and checkpoints every
  result (A13). Escalation is observable, never self-assessed (A12):
  consecutive command failures and a hard turn cap. Failed reads do
  not count — looking around is not failing
- Until delegation exists (step 7), escalation gates to the human
  with the failures attached
- control.py: start milestone now runs the executor; a stopped
  milestone gates to AWAIT_INPUT

Two bugs found by running it live against real models:

- A turn-ending gate tool (submit_plan) left a tool_call with no
  tool_result, which providers reject outright, so every planning
  turn after the first failed with a 400. The planner now records the
  gate as the call's outcome, which is also the truthful transcript
- The renderer now emits a placeholder for any tool_call lacking a
  result, so a crash between call and result (A3) cannot poison the
  transcript permanently

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 16:35:39 +00:00