Add the delegation harness (A13) — step 7b
A frontier model receives a scoped sub-task, works autonomously with full tool access, and hands back; the local loop reads the handback note and the file state, never the reasoning. - Branch at delegation start, and abort discards it. That required committing each turn's work onto the branch: without commits, deleting a branch discards nothing, so "abort discards the work" was a comforting phrase rather than a fact. The caller's own in-flight changes are committed to their branch first, so an abort cannot destroy work nobody delegated - Hard caps on turns and wall clock; a breach is a defined failure return, and its result is not usable, so the caller gates rather than resuming on a half-edited tree - Every event is checkpointed as it arrives, so a crash mid-delegation leaves the branch at a known commit and a truthful transcript - Handback is validated locally: blank fields or a non-boolean success flag are refused and the delegation continues rather than accepting a useless note Two failure modes found by running it against a real model: - A success claim with nothing changed was accepted. The handback plus file state is the entire interface back, so a note describing work that does not exist is worse than no note. Success now requires a successful mutating tool call; reporting failure still does not, since "this is impossible" is a legitimate finding - The delegate invented the absolute path /workspace, wrote there, ran its tests there, and handed back truthfully — with the repo empty and git seeing nothing. Writes outside the workspace are now refused with a message naming the real root, and both prompts state where the workspace is instead of leaving the model to guess. This is not sandboxing: work outside the repo is invisible to git, so the milestone would publish an empty PR Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1c8e7e35fc
commit
53ed11f9c4
7 files changed
+824
-7
No files matched your search
@@ -16,7 +16,7 @@ from typing import Literal
|
||||
from nisse_dev.canonical import ToolCall, ToolResult
|
||||
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import EXECUTOR_TOOLS, Workspace
|
||||
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
|
||||
|
||||
EXECUTOR_SYSTEM = """You are nisse-dev's executor, doing mechanical work.
|
||||
|
||||
@@ -30,11 +30,6 @@ plain-text summary and no tool call — that ends the milestone."""
|
||||
DEFAULT_MAX_TURNS = 24
|
||||
DEFAULT_FAILURE_LIMIT = 2
|
||||
|
||||
# Tools that change or verify something. Reading a file or listing a
|
||||
# directory is the model looking around, not doing the work, so neither
|
||||
# counts as evidence that a milestone was carried out.
|
||||
MUTATING_TOOLS = frozenset({"write_file", "run_command"})
|
||||
|
||||
# A turn with no tool call means "the plan is done". Believing that before
|
||||
# any real work has happened would record an empty milestone as a success, so
|
||||
# the claim is challenged once first.
|
||||
@@ -71,7 +66,10 @@ class LLMExecutor:
|
||||
|
||||
def run(self, milestone: int) -> ExecutorOutcome:
|
||||
model = self._roles.resolve("executor")
|
||||
system = f"{EXECUTOR_SYSTEM}\n\nYou are executing milestone {milestone}."
|
||||
system = (
|
||||
f"{EXECUTOR_SYSTEM}\n\n{self._workspace.describe()}"
|
||||
f"\n\nYou are executing milestone {milestone}."
|
||||
)
|
||||
consecutive_failures = 0
|
||||
did_work = False
|
||||
challenged = False
|
||||
|
||||
Reference in new issue
Block a user