Count only mutating tools as evidence of work
CI / checks (push) Successful in 38s
CI / checks (pull_request) Successful in 41s

The previous fix challenged a completion claim made before any tool
call, but treated every tool as work — so a model that ran one
list_dir and then declared the milestone done was believed, and CI
caught the same empty milestone again.

Reading a file or listing a directory is the model orienting itself.
Only write_file and run_command now count, so a completion claim must
be backed by something that changed or verified the tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dongarandClaude Opus 5 committed 2026-09-07 22:53:20 +00:00
1 parent 8f365263c4
commit 886cb80583
2 files changed
+26 -3

No files matched your search

+8 -2
View File
@@ -30,8 +30,13 @@ plain-text summary and no tool call — that ends the milestone."""
DEFAULT_MAX_TURNS = 24
DEFAULT_FAILURE_LIMIT = 2
# Tools that change or verify something. Reading a file or listing a
# directory is the model looking around, not doing the work, so neither
# counts as evidence that a milestone was carried out.
MUTATING_TOOLS = frozenset({"write_file", "run_command"})
# A turn with no tool call means "the plan is done". Believing that before
# any tool has been called would record an empty milestone as a success, so
# any real work has happened would record an empty milestone as a success, so
# the claim is challenged once first.
_NUDGE = (
"\n\nYou have not made any change yet, so the plan cannot be complete. "
@@ -105,7 +110,8 @@ class LLMExecutor:
)
for call in calls:
did_work = True
if call.tool in MUTATING_TOOLS:
did_work = True
outcome = self._workspace.dispatch(call.tool, call.arguments)
self._store.append_event(
ToolResult(