Put work items in the state store, and bind the log to them (M1.1)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m14s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m14s
Development Plan M1.1, as discussed. - Schema v4: a work_items table (id, scope, title, status, outcome, position, data) and an agent row whose active_item points at one (NULL: idle). Canonical events and intents carry their work item, and events their actor. Older stores are refused with a clear message: there was no production data, so no migration (the mechanism stays). - The log is reached through a Transcript bound to one work item and one actor: store.transcript(item, actor) reads only that item and writes with both. The planner, executor, delegation and expert desk get one per call from the orchestrator instead of holding the store, so a component can neither read another task's events (the planner read every earlier task's, #78) nor write without attribution. store.all_events() is kept for readers that span items (evaluation capture, dumps). Intents are begun through the transcript or with work_item=, so each belongs to its item. - The orchestrator opens a work item on 'use', closes it with its outcome (completed or abandoned), and carries what the operator said while idle into the new item as copies with their origin, so a task sent before 'use' is still planned from (D1). control.py keeps its state machine through get_state()/transition(), now over the active work item, until M2.6 retires it. - nisse_dev.dump shows work items and each event's item and actor. Tests: test_work_items.py (scoping, attribution, intent ownership, restart, outcomes, carried messages, a second task never seeing the first); the suites updated for the per-call transcript. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
1 parent
3cf4cefacd
commit
83804fe02a
40 files changed
+1026
-363
No files matched your search
@@ -99,7 +99,7 @@ test Matrix room, a budget-capped LiteLLM key; configuration in `~/nisse-e2e/`.
|
|||||||
section table this plan carried for the purpose.
|
section table this plan carried for the purpose.
|
||||||
- Acceptance: `grep` finds no reference to the old spec or its section numbers.
|
- Acceptance: `grep` finds no reference to the old spec or its section numbers.
|
||||||
|
|
||||||
**M0.2 A bring-up script.** *Pre-approved.*
|
**M0.2 A bring-up script.** *Pre-approved.* **Done.**
|
||||||
- `scripts/bringup.sh`: set aside the previous run's state, workspace and log;
|
- `scripts/bringup.sh`: set aside the previous run's state, workspace and log;
|
||||||
export the ref under test; start the agent as a systemd unit, so it outlives the
|
export the ref under test; start the agent as a systemd unit, so it outlives the
|
||||||
shell that started it; collect the log and a readable transcript
|
shell that started it; collect the log and a readable transcript
|
||||||
@@ -115,7 +115,7 @@ These fix problems found in the live runs that **survive the loop change.** Issu
|
|||||||
that only exist because of `control.py`'s state machine (#61, #79, #81) are folded
|
that only exist because of `control.py`'s state machine (#61, #79, #81) are folded
|
||||||
into Phase 2's design instead of being fixed twice.
|
into Phase 2's design instead of being fixed twice.
|
||||||
|
|
||||||
**M1.1 Work items in the state store.** *Needs discussion* (schema).
|
**M1.1 Work items in the state store.** *Needs discussion* (schema). **Done.**
|
||||||
- A work-items table with an active pointer (L1); every canonical event carries its
|
- A work-items table with an active pointer (L1); every canonical event carries its
|
||||||
work item and its actor (L7). A migration for existing stores.
|
work item and its actor (L7). A migration for existing stores.
|
||||||
- Acceptance: two work items back to back keep separate state and events; a
|
- Acceptance: two work items back to back keep separate state and events; a
|
||||||
|
|||||||
@@ -217,9 +217,7 @@ def build_agent(
|
|||||||
transport = MatrixTransport(config.matrix)
|
transport = MatrixTransport(config.matrix)
|
||||||
announce = notify or transport.send_message
|
announce = notify or transport.send_message
|
||||||
|
|
||||||
expert = ExpertDesk(
|
expert = ExpertDesk(normalized, config.roles, budget=tuning.expert.consult_budget)
|
||||||
normalized, store, config.roles, budget=tuning.expert.consult_budget
|
|
||||||
)
|
|
||||||
delegate_model = config.roles.resolve("helper")
|
delegate_model = config.roles.resolve("helper")
|
||||||
delegation = Delegation(
|
delegation = Delegation(
|
||||||
select_client(delegate_model, normalized, passthrough, catalog),
|
select_client(delegate_model, normalized, passthrough, catalog),
|
||||||
@@ -237,7 +235,6 @@ def build_agent(
|
|||||||
)
|
)
|
||||||
executor = LLMExecutor(
|
executor = LLMExecutor(
|
||||||
normalized,
|
normalized,
|
||||||
store,
|
|
||||||
config.roles,
|
config.roles,
|
||||||
workspace,
|
workspace,
|
||||||
expert=expert,
|
expert=expert,
|
||||||
@@ -272,7 +269,7 @@ def build_agent(
|
|||||||
orchestrator = Orchestrator(
|
orchestrator = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(normalized, store, config.roles, cancel=cancel),
|
LLMPlanningAgent(normalized, config.roles, cancel=cancel),
|
||||||
operators=config.operators,
|
operators=config.operators,
|
||||||
roles=config.roles,
|
roles=config.roles,
|
||||||
executor=executor,
|
executor=executor,
|
||||||
|
|||||||
@@ -168,8 +168,16 @@ def run_task(
|
|||||||
)
|
)
|
||||||
task.seed_into(workspace.root)
|
task.seed_into(workspace.root)
|
||||||
store = StateStore(root / "state.db", oplog=OpLog("nisse-bench", io.StringIO()))
|
store = StateStore(root / "state.db", oplog=OpLog("nisse-bench", io.StringIO()))
|
||||||
store.append_event(PlanSubmitted(plan=task.plan, model="benchmark"))
|
# One work item per run, as the agent's own tasks are (L1).
|
||||||
store.append_event(UserMessage(text="start milestone 1", source="benchmark"))
|
store.open_work_item(
|
||||||
|
"bench", title=task.name, position="EXECUTING", cause="benchmark run"
|
||||||
|
)
|
||||||
|
store.transcript("bench", "planner").append(
|
||||||
|
PlanSubmitted(plan=task.plan, model="benchmark")
|
||||||
|
)
|
||||||
|
store.transcript("bench", "operator").append(
|
||||||
|
UserMessage(text="start milestone 1", source="benchmark")
|
||||||
|
)
|
||||||
|
|
||||||
meter = MeteringClient(client, facts)
|
meter = MeteringClient(client, facts)
|
||||||
trims_before = getattr(client, "trimmed_requests", 0)
|
trims_before = getattr(client, "trimmed_requests", 0)
|
||||||
@@ -177,7 +185,6 @@ def run_task(
|
|||||||
roles = RoleMap(executor=model, planner=model, helper=model, expert=model)
|
roles = RoleMap(executor=model, planner=model, helper=model, expert=model)
|
||||||
executor = LLMExecutor(
|
executor = LLMExecutor(
|
||||||
meter,
|
meter,
|
||||||
store,
|
|
||||||
roles,
|
roles,
|
||||||
workspace,
|
workspace,
|
||||||
max_turns=tuning.executor.max_turns,
|
max_turns=tuning.executor.max_turns,
|
||||||
@@ -185,14 +192,14 @@ def run_task(
|
|||||||
)
|
)
|
||||||
started = time.monotonic()
|
started = time.monotonic()
|
||||||
try:
|
try:
|
||||||
outcome = executor.run(1)
|
outcome = executor.run(store.transcript("bench", "executor"), 1)
|
||||||
status, detail = outcome.status, outcome.note
|
status, detail = outcome.status, outcome.note
|
||||||
except Exception as exc: # noqa: BLE001 -- recorded, the benchmark goes on
|
except Exception as exc: # noqa: BLE001 -- recorded, the benchmark goes on
|
||||||
status, detail = "error", repr(exc)
|
status, detail = "error", repr(exc)
|
||||||
wall = time.monotonic() - started
|
wall = time.monotonic() - started
|
||||||
|
|
||||||
success, check_output = task.run_check(workspace.root)
|
success, check_output = task.run_check(workspace.root)
|
||||||
failed, invented = _count(store.read_events())
|
failed, invented = _count(store.transcript("bench", "executor").read())
|
||||||
store.close()
|
store.close()
|
||||||
|
|
||||||
return RunRecord(
|
return RunRecord(
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ record of a stop is written by the worker when the stopped work returns.
|
|||||||
import threading
|
import threading
|
||||||
|
|
||||||
from nisse_dev.canonical import ToolCall, ToolResult
|
from nisse_dev.canonical import ToolCall, ToolResult
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import Transcript
|
||||||
|
|
||||||
STOPPED_RESULT = "not run: the operator stopped the agent"
|
STOPPED_RESULT = "not run: the operator stopped the agent"
|
||||||
|
|
||||||
@@ -30,11 +30,11 @@ class Cancel:
|
|||||||
|
|
||||||
|
|
||||||
def skip_calls(
|
def skip_calls(
|
||||||
store: StateStore, calls: list[ToolCall], reason: str = STOPPED_RESULT
|
transcript: Transcript, calls: list[ToolCall], reason: str = STOPPED_RESULT
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Record calls left unrun. Every tool call needs a result for the
|
"""Record calls left unrun. Every tool call needs a result for the
|
||||||
transcript to render again, and the result says what happened."""
|
transcript to render again, and the result says what happened."""
|
||||||
for call in calls:
|
for call in calls:
|
||||||
store.append_event(
|
transcript.append(
|
||||||
ToolResult(call_id=call.call_id, result=reason, success=False)
|
ToolResult(call_id=call.call_id, result=reason, success=False)
|
||||||
)
|
)
|
||||||
@@ -112,6 +112,10 @@ class StoredEvent:
|
|||||||
id: int
|
id: int
|
||||||
ts: str
|
ts: str
|
||||||
event: CanonicalEvent
|
event: CanonicalEvent
|
||||||
|
# The work item it belongs to (None: agent-level, L6) and the role that
|
||||||
|
# produced it (L7). Rendering uses both.
|
||||||
|
work_item: str | None = None
|
||||||
|
actor: str = ""
|
||||||
|
|
||||||
|
|
||||||
def mint_call_id() -> str:
|
def mint_call_id() -> str:
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ from nisse_dev.git import GitError
|
|||||||
from nisse_dev.gitea import GiteaError
|
from nisse_dev.gitea import GiteaError
|
||||||
from nisse_dev.llm import RoleMap
|
from nisse_dev.llm import RoleMap
|
||||||
from nisse_dev.reconcile import Reconciler, reconcile_on_boot
|
from nisse_dev.reconcile import Reconciler, reconcile_on_boot
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
from nisse_dev.transport import InboundMessage, Transport
|
from nisse_dev.transport import InboundMessage, Transport
|
||||||
from nisse_dev.worker import InlineWorker, Worker
|
from nisse_dev.worker import InlineWorker, Worker
|
||||||
|
|
||||||
@@ -109,7 +109,7 @@ class MilestoneExecutor(Protocol):
|
|||||||
lives in nisse_dev.executor (imported there, not here, to keep the
|
lives in nisse_dev.executor (imported there, not here, to keep the
|
||||||
dependency pointing one way)."""
|
dependency pointing one way)."""
|
||||||
|
|
||||||
def run(self, milestone: int) -> Any: ...
|
def run(self, transcript: Transcript, milestone: int) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
class MilestonePublisher(Protocol):
|
class MilestonePublisher(Protocol):
|
||||||
@@ -149,6 +149,7 @@ class PlanBoard(Protocol):
|
|||||||
class PlanningAgent(Protocol):
|
class PlanningAgent(Protocol):
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: Transcript,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -161,6 +162,7 @@ class StubAgent:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: Transcript,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -449,8 +451,19 @@ class Orchestrator:
|
|||||||
except Exception as exc: # noqa: BLE001 -- nothing left to tell
|
except Exception as exc: # noqa: BLE001 -- nothing left to tell
|
||||||
traceback.print_exception(exc, file=sys.stderr)
|
traceback.print_exception(exc, file=sys.stderr)
|
||||||
|
|
||||||
|
def _log(self, actor: str) -> Transcript:
|
||||||
|
"""The active work item's transcript as one actor writes it; the
|
||||||
|
agent-level one when no work item is active (L6, L7)."""
|
||||||
|
item = self._store.active_item()
|
||||||
|
return self._store.transcript(item.id if item else None, actor)
|
||||||
|
|
||||||
|
def _close_item(self, outcome: str, cause: str) -> None:
|
||||||
|
item = self._store.active_item()
|
||||||
|
if item is not None and item.status == "open":
|
||||||
|
self._store.close_work_item(item.id, outcome=outcome, cause=cause)
|
||||||
|
|
||||||
def _record(self, msg: InboundMessage) -> None:
|
def _record(self, msg: InboundMessage) -> None:
|
||||||
self._store.append_event(
|
self._log("operator").append(
|
||||||
UserMessage(
|
UserMessage(
|
||||||
text=(msg.original or msg.text).strip(),
|
text=(msg.original or msg.text).strip(),
|
||||||
source=f"transport:{msg.sender}",
|
source=f"transport:{msg.sender}",
|
||||||
@@ -495,7 +508,7 @@ class Orchestrator:
|
|||||||
# Every operator message enters the canonical transcript, so planning
|
# Every operator message enters the canonical transcript, so planning
|
||||||
# turns see feedback and answers as conversation rather than as
|
# turns see feedback and answers as conversation rather than as
|
||||||
# out-of-band parameters (L7).
|
# out-of-band parameters (L7).
|
||||||
self._store.append_event(
|
self._log("operator").append(
|
||||||
UserMessage(
|
UserMessage(
|
||||||
text=(msg.original or msg.text).strip(),
|
text=(msg.original or msg.text).strip(),
|
||||||
source=f"transport:{msg.sender}",
|
source=f"transport:{msg.sender}",
|
||||||
@@ -571,6 +584,23 @@ class Orchestrator:
|
|||||||
}
|
}
|
||||||
self._project = project
|
self._project = project
|
||||||
self._store.oplog.task_id = task_id
|
self._store.oplog.task_id = task_id
|
||||||
|
# A work item of its own (L1). One left active by an unfinished task
|
||||||
|
# (reachable only through a lost pending item) is closed first.
|
||||||
|
replaced = self._store.active_item()
|
||||||
|
if replaced is not None:
|
||||||
|
self._close_item("abandoned", f"replaced by use {project}")
|
||||||
|
self._store.set_active(None, cause=f"replaced by use {project}")
|
||||||
|
self._store.open_work_item(
|
||||||
|
task_id,
|
||||||
|
title=project,
|
||||||
|
scope=project,
|
||||||
|
position=IDLE,
|
||||||
|
data=self._data,
|
||||||
|
cause=f"operator command: use {project}",
|
||||||
|
)
|
||||||
|
self._store.set_active(task_id, cause=f"operator command: use {project}")
|
||||||
|
# What the operator said before `use` is part of this task (D1).
|
||||||
|
self._store.carry_idle_messages(task_id)
|
||||||
if self._board is not None:
|
if self._board is not None:
|
||||||
try:
|
try:
|
||||||
self._data["board"] = {
|
self._data["board"] = {
|
||||||
@@ -594,7 +624,11 @@ class Orchestrator:
|
|||||||
try:
|
try:
|
||||||
with self._long_work():
|
with self._long_work():
|
||||||
action = self._agent.plan(
|
action = self._agent.plan(
|
||||||
gate_name, self._project, feedback=feedback, answer=answer
|
self._log("planner"),
|
||||||
|
gate_name,
|
||||||
|
self._project,
|
||||||
|
feedback=feedback,
|
||||||
|
answer=answer,
|
||||||
)
|
)
|
||||||
except PlanningStopped as exc:
|
except PlanningStopped as exc:
|
||||||
# Answering resumes planning with the answer, so the reason for
|
# Answering resumes planning with the answer, so the reason for
|
||||||
@@ -633,10 +667,10 @@ class Orchestrator:
|
|||||||
AWAIT_ARCH_SIGNOFF if gate_name == "arch" else AWAIT_MILESTONE_SIGNOFF
|
AWAIT_ARCH_SIGNOFF if gate_name == "arch" else AWAIT_MILESTONE_SIGNOFF
|
||||||
)
|
)
|
||||||
roles = self._data.get("roles") or {}
|
roles = self._data.get("roles") or {}
|
||||||
self._store.append_event(
|
self._log("planner").append(
|
||||||
PlanSubmitted(plan=action.content, model=roles.get("planner", "stub"))
|
PlanSubmitted(plan=action.content, model=roles.get("planner", "stub"))
|
||||||
)
|
)
|
||||||
self._store.append_event(
|
self._log("orchestrator").append(
|
||||||
GateOpened(gate=gate_name, reason=f"plan submitted (rev-{rev})")
|
GateOpened(gate=gate_name, reason=f"plan submitted (rev-{rev})")
|
||||||
)
|
)
|
||||||
self._transition(await_state, cause=f"plan submitted ({gate_name} rev-{rev})")
|
self._transition(await_state, cause=f"plan submitted ({gate_name} rev-{rev})")
|
||||||
@@ -747,7 +781,7 @@ class Orchestrator:
|
|||||||
return
|
return
|
||||||
self._transport.send_message(f"executing milestone {milestone}")
|
self._transport.send_message(f"executing milestone {milestone}")
|
||||||
with self._long_work():
|
with self._long_work():
|
||||||
outcome = self._executor.run(milestone)
|
outcome = self._executor.run(self._log("executor"), milestone)
|
||||||
if outcome.status == "stopped":
|
if outcome.status == "stopped":
|
||||||
self._halt_for_stop(
|
self._halt_for_stop(
|
||||||
what=f"milestone {milestone}",
|
what=f"milestone {milestone}",
|
||||||
@@ -826,7 +860,7 @@ class Orchestrator:
|
|||||||
):
|
):
|
||||||
return # not taken: a watch attached later still sees it
|
return # not taken: a watch attached later still sees it
|
||||||
self._take_comment(board_state, comment)
|
self._take_comment(board_state, comment)
|
||||||
self._store.append_event(
|
self._log("operator").append(
|
||||||
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
||||||
)
|
)
|
||||||
self._apply_gate_reply(
|
self._apply_gate_reply(
|
||||||
@@ -881,12 +915,12 @@ class Orchestrator:
|
|||||||
self._save(f"answering review comment {comment.id}")
|
self._save(f"answering review comment {comment.id}")
|
||||||
# The comment enters the transcript, so the executor sees review
|
# The comment enters the transcript, so the executor sees review
|
||||||
# feedback as conversation rather than as an out-of-band instruction.
|
# feedback as conversation rather than as an out-of-band instruction.
|
||||||
self._store.append_event(
|
self._log("operator").append(
|
||||||
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
with self._long_work():
|
with self._long_work():
|
||||||
outcome = self._executor.run(milestone)
|
outcome = self._executor.run(self._log("executor"), milestone)
|
||||||
except Exception:
|
except Exception:
|
||||||
# Reported to the operator by _settled: answered, if not well.
|
# Reported to the operator by _settled: answered, if not well.
|
||||||
self._done_reviewing(pr)
|
self._done_reviewing(pr)
|
||||||
@@ -1002,6 +1036,7 @@ class Orchestrator:
|
|||||||
pr = self._data.get("pr") or {}
|
pr = self._data.get("pr") or {}
|
||||||
project = self._project
|
project = self._project
|
||||||
self.stop_watching()
|
self.stop_watching()
|
||||||
|
self._close_item("abandoned", f"operator: abandon ({sender})")
|
||||||
self._data = {**_fresh_data()}
|
self._data = {**_fresh_data()}
|
||||||
self._project = None
|
self._project = None
|
||||||
self._store.oplog.task_id = None
|
self._store.oplog.task_id = None
|
||||||
@@ -1084,6 +1119,7 @@ class Orchestrator:
|
|||||||
elif command == "project complete":
|
elif command == "project complete":
|
||||||
board_state = self._data.get("board") or {}
|
board_state = self._data.get("board") or {}
|
||||||
self.stop_watching()
|
self.stop_watching()
|
||||||
|
self._close_item("completed", "operator: project complete")
|
||||||
self._data = {**_fresh_data()}
|
self._data = {**_fresh_data()}
|
||||||
self._project = None
|
self._project = None
|
||||||
self._store.oplog.task_id = None
|
self._store.oplog.task_id = None
|
||||||
|
|||||||
@@ -33,13 +33,14 @@ from nisse_dev.canonical import (
|
|||||||
AssistantText,
|
AssistantText,
|
||||||
DelegationCompleted,
|
DelegationCompleted,
|
||||||
DelegationStarted,
|
DelegationStarted,
|
||||||
|
StoredEvent,
|
||||||
ToolCall,
|
ToolCall,
|
||||||
ToolResult,
|
ToolResult,
|
||||||
)
|
)
|
||||||
from nisse_dev.git import GitError, GitRepo
|
from nisse_dev.git import GitError, GitRepo
|
||||||
from nisse_dev.llm import LLMClient, LLMError, UpstreamUnavailable
|
from nisse_dev.llm import LLMClient, LLMError, UpstreamUnavailable
|
||||||
from nisse_dev.reconcile import Reconciler
|
from nisse_dev.reconcile import Reconciler
|
||||||
from nisse_dev.state import Intent, StateStore
|
from nisse_dev.state import Intent, StateStore, Transcript
|
||||||
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
|
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
|
||||||
|
|
||||||
DELEGATION_INTENT = "delegation"
|
DELEGATION_INTENT = "delegation"
|
||||||
@@ -184,12 +185,16 @@ class Delegation:
|
|||||||
# a reason, and the next step is to talk about that reason.
|
# a reason, and the next step is to talk about that reason.
|
||||||
self._cancel = cancel or Cancel()
|
self._cancel = cancel or Cancel()
|
||||||
|
|
||||||
def run(self, *, brief: str, branch: str, role: str = "helper") -> DelegationResult:
|
def run(
|
||||||
|
self, transcript: Transcript, *, brief: str, branch: str, role: str = "helper"
|
||||||
|
) -> DelegationResult:
|
||||||
|
"""Run a delegation in the work item whose transcript this is,
|
||||||
|
writing as the helper (L6, L7); its intent is owned by that item."""
|
||||||
started_at = self._now()
|
started_at = self._now()
|
||||||
base_branch = self._git.current_branch()
|
base_branch = self._git.current_branch()
|
||||||
# Recorded before the branch exists, so a crash anywhere after this
|
# Recorded before the branch exists, so a crash anywhere after this
|
||||||
# leaves a durable statement of what was in flight (L8).
|
# leaves a durable statement of what was in flight (L8).
|
||||||
intent_id = self._store.begin_intent(
|
intent_id = transcript.begin_intent(
|
||||||
DELEGATION_INTENT,
|
DELEGATION_INTENT,
|
||||||
{
|
{
|
||||||
"branch": branch,
|
"branch": branch,
|
||||||
@@ -210,7 +215,7 @@ class Delegation:
|
|||||||
self._store.abort_intent(intent_id, cause=f"could not branch: {exc}")
|
self._store.abort_intent(intent_id, cause=f"could not branch: {exc}")
|
||||||
return DelegationResult("failed", f"could not branch for delegation: {exc}")
|
return DelegationResult("failed", f"could not branch for delegation: {exc}")
|
||||||
|
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
DelegationStarted(
|
DelegationStarted(
|
||||||
brief=brief,
|
brief=brief,
|
||||||
role=role,
|
role=role,
|
||||||
@@ -219,9 +224,9 @@ class Delegation:
|
|||||||
backing_model=self._backing_model,
|
backing_model=self._backing_model,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
result = self._loop(brief, branch, started_at)
|
result = self._loop(transcript, brief, branch, started_at)
|
||||||
result.duration_s = self._now() - started_at
|
result.duration_s = self._now() - started_at
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback=result.handback.render() if result.handback else result.note,
|
handback=result.handback.render() if result.handback else result.note,
|
||||||
artifacts=result.artifacts,
|
artifacts=result.artifacts,
|
||||||
@@ -256,6 +261,7 @@ class Delegation:
|
|||||||
"""
|
"""
|
||||||
payload = intent.payload
|
payload = intent.payload
|
||||||
branch = str(payload["branch"])
|
branch = str(payload["branch"])
|
||||||
|
transcript = self._store.transcript(intent.work_item, "helper")
|
||||||
started_at = self._now()
|
started_at = self._now()
|
||||||
try:
|
try:
|
||||||
self._git.checkout_existing_branch(branch)
|
self._git.checkout_existing_branch(branch)
|
||||||
@@ -265,9 +271,11 @@ class Delegation:
|
|||||||
)
|
)
|
||||||
return DelegationResult("failed", f"could not resume on {branch}: {exc}")
|
return DelegationResult("failed", f"could not resume on {branch}: {exc}")
|
||||||
|
|
||||||
result = self._loop(str(payload.get("brief", "")), branch, started_at)
|
result = self._loop(
|
||||||
|
transcript, str(payload.get("brief", "")), branch, started_at
|
||||||
|
)
|
||||||
result.duration_s = self._now() - started_at
|
result.duration_s = self._now() - started_at
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback=result.handback.render() if result.handback else result.note,
|
handback=result.handback.render() if result.handback else result.note,
|
||||||
artifacts=result.artifacts,
|
artifacts=result.artifacts,
|
||||||
@@ -310,7 +318,9 @@ class Delegation:
|
|||||||
return self.resume(intent)
|
return self.resume(intent)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def _loop(self, brief: str, branch: str, started_at: float) -> DelegationResult:
|
def _loop(
|
||||||
|
self, transcript: Transcript, brief: str, branch: str, started_at: float
|
||||||
|
) -> DelegationResult:
|
||||||
system = (
|
system = (
|
||||||
f"{SYSTEM_PROMPT}\n\n{self._workspace.describe()}"
|
f"{SYSTEM_PROMPT}\n\n{self._workspace.describe()}"
|
||||||
f"\n\nYour sub-task:\n{brief}"
|
f"\n\nYour sub-task:\n{brief}"
|
||||||
@@ -340,7 +350,7 @@ class Delegation:
|
|||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
produced = self._client.complete(
|
produced = self._client.complete(
|
||||||
self._store.read_events(),
|
transcript.read(),
|
||||||
tools=DELEGATE_TOOLS,
|
tools=DELEGATE_TOOLS,
|
||||||
model=self._model,
|
model=self._model,
|
||||||
system=system,
|
system=system,
|
||||||
@@ -380,7 +390,7 @@ class Delegation:
|
|||||||
for event in produced:
|
for event in produced:
|
||||||
# Checkpoint as it arrives, never batched at handback (L9):
|
# Checkpoint as it arrives, never batched at handback (L9):
|
||||||
# this is what makes the delegation resumable after a crash.
|
# this is what makes the delegation resumable after a crash.
|
||||||
self._store.append_event(event)
|
transcript.append(event)
|
||||||
if isinstance(event, ToolCall):
|
if isinstance(event, ToolCall):
|
||||||
calls.append(event)
|
calls.append(event)
|
||||||
|
|
||||||
@@ -400,7 +410,7 @@ class Delegation:
|
|||||||
|
|
||||||
for index, call in enumerate(calls):
|
for index, call in enumerate(calls):
|
||||||
if self._cancel.requested:
|
if self._cancel.requested:
|
||||||
skip_calls(self._store, calls[index:])
|
skip_calls(transcript, calls[index:])
|
||||||
return DelegationResult(
|
return DelegationResult(
|
||||||
"stopped",
|
"stopped",
|
||||||
f"stopped by the operator during turn {turn + 1}",
|
f"stopped by the operator during turn {turn + 1}",
|
||||||
@@ -415,7 +425,7 @@ class Delegation:
|
|||||||
# and the file state are the whole interface back, so
|
# and the file state are the whole interface back, so
|
||||||
# an unbacked claim would hand the caller a note about
|
# an unbacked claim would hand the caller a note about
|
||||||
# work that does not exist.
|
# work that does not exist.
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result=(
|
result=(
|
||||||
@@ -429,7 +439,7 @@ class Delegation:
|
|||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
if handback is None:
|
if handback is None:
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result=(
|
result=(
|
||||||
@@ -440,7 +450,7 @@ class Delegation:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result="handback accepted; delegation closed",
|
result="handback accepted; delegation closed",
|
||||||
@@ -463,7 +473,7 @@ class Delegation:
|
|||||||
path = call.arguments.get("path")
|
path = call.arguments.get("path")
|
||||||
if isinstance(path, str):
|
if isinstance(path, str):
|
||||||
touched.add(path)
|
touched.add(path)
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result=outcome.result,
|
result=outcome.result,
|
||||||
@@ -552,11 +562,12 @@ def delegation_verdict(
|
|||||||
branch = intent.payload.get("branch")
|
branch = intent.payload.get("branch")
|
||||||
if not branch:
|
if not branch:
|
||||||
return "gate"
|
return "gate"
|
||||||
if _completed_after_start(store):
|
# The delegation's own work item: an intent is owned by the item it ran in.
|
||||||
|
events = store.transcript(intent.work_item, "helper").read()
|
||||||
|
if _completed_after_start(events):
|
||||||
return "settled" # it finished; the intent merely outlived its commit
|
return "settled" # it finished; the intent merely outlived its commit
|
||||||
if not git_repo.branch_exists(str(branch)):
|
if not git_repo.branch_exists(str(branch)):
|
||||||
return "absent" # the crash landed before any work existed
|
return "absent" # the crash landed before any work existed
|
||||||
events = store.read_events()
|
|
||||||
if not events:
|
if not events:
|
||||||
return "gate"
|
return "gate"
|
||||||
last = events[-1].event
|
last = events[-1].event
|
||||||
@@ -569,9 +580,9 @@ def delegation_verdict(
|
|||||||
return "resume"
|
return "resume"
|
||||||
|
|
||||||
|
|
||||||
def _completed_after_start(store: StateStore) -> bool:
|
def _completed_after_start(events: list[StoredEvent]) -> bool:
|
||||||
"""True when the last delegation event in the log is a completion."""
|
"""True when the last delegation event in the log is a completion."""
|
||||||
for stored in reversed(store.read_events()):
|
for stored in reversed(events):
|
||||||
if isinstance(stored.event, DelegationCompleted):
|
if isinstance(stored.event, DelegationCompleted):
|
||||||
return True
|
return True
|
||||||
if isinstance(stored.event, DelegationStarted):
|
if isinstance(stored.event, DelegationStarted):
|
||||||
|
|||||||
@@ -3,9 +3,10 @@
|
|||||||
python -m nisse_dev.dump STATE_DB [--full]
|
python -m nisse_dev.dump STATE_DB [--full]
|
||||||
|
|
||||||
Opens the store read-only (SQLite `mode=ro`), so it is safe to run against a
|
Opens the store read-only (SQLite `mode=ro`), so it is safe to run against a
|
||||||
live agent's database: it never migrates, writes or locks it. Prints the
|
live agent's database: it never migrates, writes or locks it. Prints the work
|
||||||
agent's state, pending intents, and the canonical log, one event per line;
|
items and which is active, pending intents, and the canonical log, one event
|
||||||
long texts are shortened unless `--full` is given.
|
per line with its work item and actor; long texts are shortened unless
|
||||||
|
`--full` is given.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
@@ -55,27 +56,34 @@ def _event_line(kind: str, payload: dict[str, Any], full: bool) -> str:
|
|||||||
def render(path: Path, *, full: bool = False) -> str:
|
def render(path: Path, *, full: bool = False) -> str:
|
||||||
conn = _connect(path)
|
conn = _connect(path)
|
||||||
lines: list[str] = []
|
lines: list[str] = []
|
||||||
state = conn.execute("SELECT * FROM agent_state WHERE id = 1").fetchone()
|
agent = conn.execute("SELECT * FROM agent WHERE id = 1").fetchone()
|
||||||
if state is None:
|
active = agent["active_item"] if agent else None
|
||||||
lines.append("state: (none)")
|
lines.append(f"active work item: {active or '(none: idle)'}")
|
||||||
else:
|
for item in conn.execute("SELECT * FROM work_items ORDER BY opened_at"):
|
||||||
data = json.loads(state["data"])
|
data = json.loads(item["data"])
|
||||||
gate = data.get("gate") or {}
|
gate = data.get("gate") or {}
|
||||||
|
status = item["status"] + (f" ({item['outcome']})" if item["outcome"] else "")
|
||||||
lines.append(
|
lines.append(
|
||||||
f"state: {state['state']} project: {state['project']} "
|
f" {item['id']} {status} position: {item['position']} "
|
||||||
f"task: {data.get('task_id')} gate: "
|
f"scope: {item['scope']} gate: "
|
||||||
+ (f"{gate.get('name')} rev-{gate.get('rev')}" if gate else "none")
|
+ (f"{gate.get('name')} rev-{gate.get('rev')}" if gate else "none")
|
||||||
)
|
)
|
||||||
pending = conn.execute(
|
pending = conn.execute(
|
||||||
"SELECT id, kind, payload FROM intent_log WHERE status = 'pending'"
|
"SELECT id, kind, payload, work_item FROM intent_log WHERE status = 'pending'"
|
||||||
).fetchall()
|
).fetchall()
|
||||||
for row in pending:
|
for row in pending:
|
||||||
lines.append(f"pending intent #{row['id']} {row['kind']}: {row['payload']}")
|
lines.append(
|
||||||
|
f"pending intent #{row['id']} {row['kind']} "
|
||||||
|
f"[{row['work_item'] or 'agent'}]: {row['payload']}"
|
||||||
|
)
|
||||||
lines.append("")
|
lines.append("")
|
||||||
for row in conn.execute("SELECT * FROM canonical_events ORDER BY id"):
|
for row in conn.execute("SELECT * FROM canonical_events ORDER BY id"):
|
||||||
payload = json.loads(row["payload"])
|
payload = json.loads(row["payload"])
|
||||||
time = row["ts"][11:19]
|
time = row["ts"][11:19]
|
||||||
lines.append(f"#{row['id']} {time} {_event_line(row['type'], payload, full)}")
|
where = f"[{row['work_item'] or 'agent'}/{row['actor']}]"
|
||||||
|
lines.append(
|
||||||
|
f"#{row['id']} {time} {where} {_event_line(row['type'], payload, full)}"
|
||||||
|
)
|
||||||
conn.close()
|
conn.close()
|
||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ def collect(store: StateStore, after: int = 0) -> list[EvalRecord]:
|
|||||||
verdicts = store.outcomes()
|
verdicts = store.outcomes()
|
||||||
records: list[EvalRecord] = []
|
records: list[EvalRecord] = []
|
||||||
pending: tuple[int, DelegationStarted] | None = None
|
pending: tuple[int, DelegationStarted] | None = None
|
||||||
for stored in store.read_events(after=after):
|
for stored in store.all_events(after=after):
|
||||||
if isinstance(stored.event, DelegationStarted):
|
if isinstance(stored.event, DelegationStarted):
|
||||||
pending = (stored.id, stored.event)
|
pending = (stored.id, stored.event)
|
||||||
elif isinstance(stored.event, DelegationCompleted) and pending is not None:
|
elif isinstance(stored.event, DelegationCompleted) and pending is not None:
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from nisse_dev.canonical import PlanSubmitted, StoredEvent, ToolCall, ToolResult
|
|||||||
from nisse_dev.delegation import Delegation
|
from nisse_dev.delegation import Delegation
|
||||||
from nisse_dev.expert import ASK_EXPERT_TOOL, BudgetExhausted, ExpertDesk
|
from nisse_dev.expert import ASK_EXPERT_TOOL, BudgetExhausted, ExpertDesk
|
||||||
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import Transcript
|
||||||
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, ToolOutcome, Workspace
|
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, ToolOutcome, Workspace
|
||||||
|
|
||||||
EXECUTOR_SYSTEM = """You are nisse-dev's executor, doing mechanical work.
|
EXECUTOR_SYSTEM = """You are nisse-dev's executor, doing mechanical work.
|
||||||
@@ -76,7 +76,6 @@ class LLMExecutor:
|
|||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
client: LLMClient,
|
client: LLMClient,
|
||||||
store: StateStore,
|
|
||||||
roles: RoleMap,
|
roles: RoleMap,
|
||||||
workspace: Workspace,
|
workspace: Workspace,
|
||||||
*,
|
*,
|
||||||
@@ -88,7 +87,6 @@ class LLMExecutor:
|
|||||||
cancel: Cancel | None = None,
|
cancel: Cancel | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._client = client
|
self._client = client
|
||||||
self._store = store
|
|
||||||
self._roles = roles
|
self._roles = roles
|
||||||
self._workspace = workspace
|
self._workspace = workspace
|
||||||
self._max_turns = max_turns
|
self._max_turns = max_turns
|
||||||
@@ -113,7 +111,7 @@ class LLMExecutor:
|
|||||||
self._notify(f"turn {turn + 1}: {summary}")
|
self._notify(f"turn {turn + 1}: {summary}")
|
||||||
|
|
||||||
def _escalate(
|
def _escalate(
|
||||||
self, milestone: int, failures: int, last_failure: str
|
self, transcript: Transcript, milestone: int, failures: int, last_failure: str
|
||||||
) -> ExecutorOutcome | None:
|
) -> ExecutorOutcome | None:
|
||||||
"""Hand the sub-task to a stronger model, per L9.
|
"""Hand the sub-task to a stronger model, per L9.
|
||||||
|
|
||||||
@@ -134,7 +132,9 @@ class LLMExecutor:
|
|||||||
"above. Finish this sub-task, staying inside the approved plan."
|
"above. Finish this sub-task, staying inside the approved plan."
|
||||||
)
|
)
|
||||||
result = self._delegate.run(
|
result = self._delegate.run(
|
||||||
brief=brief, branch=f"delegation/milestone-{milestone}-{failures}"
|
transcript.as_actor("helper"),
|
||||||
|
brief=brief,
|
||||||
|
branch=f"delegation/milestone-{milestone}-{failures}",
|
||||||
)
|
)
|
||||||
if result.usable:
|
if result.usable:
|
||||||
return None
|
return None
|
||||||
@@ -153,13 +153,16 @@ class LLMExecutor:
|
|||||||
return [*EXECUTOR_TOOLS, FINISH_TOOL]
|
return [*EXECUTOR_TOOLS, FINISH_TOOL]
|
||||||
return [*EXECUTOR_TOOLS, ASK_EXPERT_TOOL, FINISH_TOOL]
|
return [*EXECUTOR_TOOLS, ASK_EXPERT_TOOL, FINISH_TOOL]
|
||||||
|
|
||||||
def _consult(self, arguments: dict[str, Any]) -> ToolOutcome:
|
def _consult(
|
||||||
|
self, transcript: Transcript, arguments: dict[str, Any]
|
||||||
|
) -> ToolOutcome:
|
||||||
assert self._expert is not None
|
assert self._expert is not None
|
||||||
question = str(arguments.get("question", "")).strip()
|
question = str(arguments.get("question", "")).strip()
|
||||||
if not question:
|
if not question:
|
||||||
return ToolOutcome("ask_expert needs a question", success=False)
|
return ToolOutcome("ask_expert needs a question", success=False)
|
||||||
try:
|
try:
|
||||||
return ToolOutcome(self._expert.ask(question).render(), success=True)
|
answer = self._expert.ask(transcript.as_actor("expert"), question)
|
||||||
|
return ToolOutcome(answer.render(), success=True)
|
||||||
except BudgetExhausted as exc:
|
except BudgetExhausted as exc:
|
||||||
# Not an error the model should route around: it is the signal
|
# Not an error the model should route around: it is the signal
|
||||||
# that it is circling and should get on with the work or fail.
|
# that it is circling and should get on with the work or fail.
|
||||||
@@ -167,7 +170,7 @@ class LLMExecutor:
|
|||||||
except LLMError as exc:
|
except LLMError as exc:
|
||||||
return ToolOutcome(f"the expert did not answer: {exc}", success=False)
|
return ToolOutcome(f"the expert did not answer: {exc}", success=False)
|
||||||
|
|
||||||
def _scope(self) -> tuple[str | None, int]:
|
def _scope(self, transcript: Transcript) -> tuple[str | None, int]:
|
||||||
"""The approved plan, and where this milestone's events begin.
|
"""The approved plan, and where this milestone's events begin.
|
||||||
|
|
||||||
The most recent plan in the log is the approved milestone plan: no
|
The most recent plan in the log is the approved milestone plan: no
|
||||||
@@ -177,7 +180,7 @@ class LLMExecutor:
|
|||||||
context window cannot afford and the plan already distils.
|
context window cannot afford and the plan already distils.
|
||||||
"""
|
"""
|
||||||
latest: StoredEvent | None = None
|
latest: StoredEvent | None = None
|
||||||
for stored in self._store.read_events():
|
for stored in transcript.read():
|
||||||
if isinstance(stored.event, PlanSubmitted):
|
if isinstance(stored.event, PlanSubmitted):
|
||||||
latest = stored
|
latest = stored
|
||||||
if latest is None:
|
if latest is None:
|
||||||
@@ -185,9 +188,11 @@ class LLMExecutor:
|
|||||||
assert isinstance(latest.event, PlanSubmitted)
|
assert isinstance(latest.event, PlanSubmitted)
|
||||||
return latest.event.plan, latest.id
|
return latest.event.plan, latest.id
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: Transcript, milestone: int) -> ExecutorOutcome:
|
||||||
|
"""Run a milestone of the work item whose transcript this is, writing
|
||||||
|
as the executor (L6, L7)."""
|
||||||
model = self._roles.resolve("executor")
|
model = self._roles.resolve("executor")
|
||||||
plan, after = self._scope()
|
plan, after = self._scope(transcript)
|
||||||
system = (
|
system = (
|
||||||
f"{EXECUTOR_SYSTEM}\n\n{self._workspace.describe()}"
|
f"{EXECUTOR_SYSTEM}\n\n{self._workspace.describe()}"
|
||||||
f"\n\nYou are executing milestone {milestone}."
|
f"\n\nYou are executing milestone {milestone}."
|
||||||
@@ -209,7 +214,7 @@ class LLMExecutor:
|
|||||||
return ExecutorOutcome("stopped", f"before turn {turn + 1}")
|
return ExecutorOutcome("stopped", f"before turn {turn + 1}")
|
||||||
try:
|
try:
|
||||||
produced = self._client.complete(
|
produced = self._client.complete(
|
||||||
self._store.read_events(after=after),
|
transcript.read(after=after),
|
||||||
tools=tools,
|
tools=tools,
|
||||||
model=model,
|
model=model,
|
||||||
system=system,
|
system=system,
|
||||||
@@ -219,7 +224,7 @@ class LLMExecutor:
|
|||||||
|
|
||||||
calls: list[ToolCall] = []
|
calls: list[ToolCall] = []
|
||||||
for event in produced:
|
for event in produced:
|
||||||
self._store.append_event(event)
|
transcript.append(event)
|
||||||
if isinstance(event, ToolCall):
|
if isinstance(event, ToolCall):
|
||||||
calls.append(event)
|
calls.append(event)
|
||||||
|
|
||||||
@@ -235,7 +240,7 @@ class LLMExecutor:
|
|||||||
self._report(turn, calls)
|
self._report(turn, calls)
|
||||||
for index, call in enumerate(calls):
|
for index, call in enumerate(calls):
|
||||||
if self._cancel.requested:
|
if self._cancel.requested:
|
||||||
skip_calls(self._store, calls[index:])
|
skip_calls(transcript, calls[index:])
|
||||||
return ExecutorOutcome("stopped", f"during turn {turn + 1}")
|
return ExecutorOutcome("stopped", f"during turn {turn + 1}")
|
||||||
if call.tool == "finish":
|
if call.tool == "finish":
|
||||||
if not did_work and not finish_refused:
|
if not did_work and not finish_refused:
|
||||||
@@ -248,7 +253,7 @@ class LLMExecutor:
|
|||||||
"making any change",
|
"making any change",
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result="milestone finished",
|
result="milestone finished",
|
||||||
@@ -256,7 +261,7 @@ class LLMExecutor:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
skip_calls(
|
skip_calls(
|
||||||
self._store,
|
transcript,
|
||||||
calls[index + 1 :],
|
calls[index + 1 :],
|
||||||
"not run: finish had already ended the milestone",
|
"not run: finish had already ended the milestone",
|
||||||
)
|
)
|
||||||
@@ -274,12 +279,12 @@ class LLMExecutor:
|
|||||||
success=False,
|
success=False,
|
||||||
)
|
)
|
||||||
elif call.tool == "ask_expert":
|
elif call.tool == "ask_expert":
|
||||||
outcome = self._consult(call.arguments)
|
outcome = self._consult(transcript, call.arguments)
|
||||||
else:
|
else:
|
||||||
if call.tool in MUTATING_TOOLS:
|
if call.tool in MUTATING_TOOLS:
|
||||||
did_work = True
|
did_work = True
|
||||||
outcome = self._workspace.dispatch(call.tool, call.arguments)
|
outcome = self._workspace.dispatch(call.tool, call.arguments)
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=call.call_id,
|
call_id=call.call_id,
|
||||||
result=outcome.result,
|
result=outcome.result,
|
||||||
@@ -298,6 +303,7 @@ class LLMExecutor:
|
|||||||
# Escalating to the human is the fallback when there
|
# Escalating to the human is the fallback when there
|
||||||
# is no delegate, not the intended path.
|
# is no delegate, not the intended path.
|
||||||
escalation = self._escalate(
|
escalation = self._escalate(
|
||||||
|
transcript,
|
||||||
milestone,
|
milestone,
|
||||||
consecutive_failures,
|
consecutive_failures,
|
||||||
outcome.result,
|
outcome.result,
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ from nisse_dev.canonical import (
|
|||||||
UserMessage,
|
UserMessage,
|
||||||
)
|
)
|
||||||
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import Transcript
|
||||||
|
|
||||||
EXPERT_ROLE = "expert"
|
EXPERT_ROLE = "expert"
|
||||||
DEFAULT_BUDGET = 3
|
DEFAULT_BUDGET = 3
|
||||||
@@ -97,32 +97,34 @@ class ExpertDesk:
|
|||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
client: LLMClient,
|
client: LLMClient,
|
||||||
store: StateStore,
|
|
||||||
roles: RoleMap,
|
roles: RoleMap,
|
||||||
*,
|
*,
|
||||||
budget: int = DEFAULT_BUDGET,
|
budget: int = DEFAULT_BUDGET,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._client = client
|
self._client = client
|
||||||
self._store = store
|
|
||||||
self._roles = roles
|
self._roles = roles
|
||||||
self._budget = budget
|
self._budget = budget
|
||||||
|
|
||||||
def consults_used(self, since_event_id: int = 0) -> int:
|
def consults_used(self, transcript: Transcript, since_event_id: int = 0) -> int:
|
||||||
return sum(
|
return sum(
|
||||||
1
|
1
|
||||||
for e in self._store.read_events(after=since_event_id)
|
for e in transcript.read(after=since_event_id)
|
||||||
if isinstance(e.event, DelegationStarted) and e.event.role == EXPERT_ROLE
|
if isinstance(e.event, DelegationStarted) and e.event.role == EXPERT_ROLE
|
||||||
)
|
)
|
||||||
|
|
||||||
def ask(self, question: str, *, since_event_id: int = 0) -> ExpertAnswer:
|
def ask(
|
||||||
used = self.consults_used(since_event_id)
|
self, transcript: Transcript, question: str, *, since_event_id: int = 0
|
||||||
|
) -> ExpertAnswer:
|
||||||
|
"""The consult is recorded in the work item's transcript, as the
|
||||||
|
expert, and the budget counts that work item's consults (L9)."""
|
||||||
|
used = self.consults_used(transcript, since_event_id)
|
||||||
if used >= self._budget:
|
if used >= self._budget:
|
||||||
raise BudgetExhausted(
|
raise BudgetExhausted(
|
||||||
f"consult budget of {self._budget} is spent for this task "
|
f"consult budget of {self._budget} is spent for this task "
|
||||||
f"({used} used); escalate or ask the operator instead"
|
f"({used} used); escalate or ask the operator instead"
|
||||||
)
|
)
|
||||||
model = self._roles.resolve(EXPERT_ROLE)
|
model = self._roles.resolve(EXPERT_ROLE)
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
DelegationStarted(
|
DelegationStarted(
|
||||||
brief=question,
|
brief=question,
|
||||||
role=EXPERT_ROLE,
|
role=EXPERT_ROLE,
|
||||||
@@ -131,7 +133,7 @@ class ExpertDesk:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
answer, note = self._answer(question, model)
|
answer, note = self._answer(question, model)
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback=answer.render() if answer else note,
|
handback=answer.render() if answer else note,
|
||||||
artifacts=[],
|
artifacts=[],
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ from nisse_dev.cancel import Cancel
|
|||||||
from nisse_dev.canonical import AssistantText, ToolCall, ToolResult
|
from nisse_dev.canonical import AssistantText, ToolCall, ToolResult
|
||||||
from nisse_dev.control import AskHuman, PlanningError, PlanningStopped, SubmitPlan
|
from nisse_dev.control import AskHuman, PlanningError, PlanningStopped, SubmitPlan
|
||||||
from nisse_dev.llm import BudgetExhausted, LLMClient, LLMError, RoleMap
|
from nisse_dev.llm import BudgetExhausted, LLMClient, LLMError, RoleMap
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import Transcript
|
||||||
|
|
||||||
# Plans may be executed by a materially weaker model, which cannot
|
# Plans may be executed by a materially weaker model, which cannot
|
||||||
# reconstruct intent. This predates the architecture's capability profiles
|
# reconstruct intent. This predates the architecture's capability profiles
|
||||||
@@ -83,14 +83,12 @@ class LLMPlanningAgent:
|
|||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
client: LLMClient,
|
client: LLMClient,
|
||||||
store: StateStore,
|
|
||||||
roles: RoleMap,
|
roles: RoleMap,
|
||||||
*,
|
*,
|
||||||
system_prompt: str = SYSTEM_PROMPT,
|
system_prompt: str = SYSTEM_PROMPT,
|
||||||
cancel: Cancel | None = None,
|
cancel: Cancel | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._client = client
|
self._client = client
|
||||||
self._store = store
|
|
||||||
self._roles = roles
|
self._roles = roles
|
||||||
self._system_prompt = system_prompt
|
self._system_prompt = system_prompt
|
||||||
# Checked before each turn only. A turn is one model call, and a turn
|
# Checked before each turn only. A turn is one model call, and a turn
|
||||||
@@ -99,6 +97,7 @@ class LLMPlanningAgent:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: Transcript,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -106,18 +105,21 @@ class LLMPlanningAgent:
|
|||||||
) -> SubmitPlan | AskHuman:
|
) -> SubmitPlan | AskHuman:
|
||||||
"""Operator feedback and answers are already durable in the canonical
|
"""Operator feedback and answers are already durable in the canonical
|
||||||
log (the orchestrator appends every inbound message), so they are not
|
log (the orchestrator appends every inbound message), so they are not
|
||||||
re-injected here; the log is the single source of context."""
|
re-injected here; the log is the single source of context. The
|
||||||
|
transcript is the work item's, written as the planner (L6, L7)."""
|
||||||
model = self._roles.resolve("planner")
|
model = self._roles.resolve("planner")
|
||||||
instruction = (
|
instruction = (
|
||||||
f"{self._system_prompt}\n\n"
|
f"{self._system_prompt}\n\n"
|
||||||
f"Current task: draft the {gate} plan for project {project!r}."
|
f"Current task: draft the {gate} plan for project {project!r}."
|
||||||
)
|
)
|
||||||
self._check_stop()
|
self._check_stop()
|
||||||
action, first_error = self._try_turn(model, instruction)
|
action, first_error = self._try_turn(transcript, model, instruction)
|
||||||
if action is not None:
|
if action is not None:
|
||||||
return action
|
return action
|
||||||
self._check_stop()
|
self._check_stop()
|
||||||
action, second_error = self._try_turn(model, f"{instruction}\n\n{_NUDGE}")
|
action, second_error = self._try_turn(
|
||||||
|
transcript, model, f"{instruction}\n\n{_NUDGE}"
|
||||||
|
)
|
||||||
if action is None:
|
if action is None:
|
||||||
detail = second_error or first_error
|
detail = second_error or first_error
|
||||||
raise PlanningError(
|
raise PlanningError(
|
||||||
@@ -131,13 +133,13 @@ class LLMPlanningAgent:
|
|||||||
raise PlanningStopped("stopped by the operator")
|
raise PlanningStopped("stopped by the operator")
|
||||||
|
|
||||||
def _try_turn(
|
def _try_turn(
|
||||||
self, model: str, system: str
|
self, transcript: Transcript, model: str, system: str
|
||||||
) -> tuple[SubmitPlan | AskHuman | None, str | None]:
|
) -> tuple[SubmitPlan | AskHuman | None, str | None]:
|
||||||
"""A malformed or failed call is a bad turn, not a crash: the caller
|
"""A malformed or failed call is a bad turn, not a crash: the caller
|
||||||
retries once and then gates to the human (L9 — judgment about model
|
retries once and then gates to the human (L9 — judgment about model
|
||||||
output is observable, never the model's own claim)."""
|
output is observable, never the model's own claim)."""
|
||||||
try:
|
try:
|
||||||
return self._turn(model, system), None
|
return self._turn(transcript, model, system), None
|
||||||
except BudgetExhausted:
|
except BudgetExhausted:
|
||||||
# Retrying cannot help: the budget will not reappear within a
|
# Retrying cannot help: the budget will not reappear within a
|
||||||
# turn. Surface it now so the operator is told rather than the
|
# turn. Surface it now so the operator is told rather than the
|
||||||
@@ -146,8 +148,10 @@ class LLMPlanningAgent:
|
|||||||
except LLMError as exc:
|
except LLMError as exc:
|
||||||
return None, str(exc)
|
return None, str(exc)
|
||||||
|
|
||||||
def _turn(self, model: str, system: str) -> SubmitPlan | AskHuman | None:
|
def _turn(
|
||||||
events = self._store.read_events()
|
self, transcript: Transcript, model: str, system: str
|
||||||
|
) -> SubmitPlan | AskHuman | None:
|
||||||
|
events = transcript.read()
|
||||||
produced = self._client.complete(
|
produced = self._client.complete(
|
||||||
events, tools=PLANNER_TOOLS, model=model, system=system
|
events, tools=PLANNER_TOOLS, model=model, system=system
|
||||||
)
|
)
|
||||||
@@ -155,7 +159,7 @@ class LLMPlanningAgent:
|
|||||||
for event in produced:
|
for event in produced:
|
||||||
# Append before interpreting: the transcript must be truthful even
|
# Append before interpreting: the transcript must be truthful even
|
||||||
# if the call below rejects the arguments.
|
# if the call below rejects the arguments.
|
||||||
self._store.append_event(event)
|
transcript.append(event)
|
||||||
if isinstance(event, AssistantText) or action is not None:
|
if isinstance(event, AssistantText) or action is not None:
|
||||||
continue
|
continue
|
||||||
if isinstance(event, ToolCall):
|
if isinstance(event, ToolCall):
|
||||||
@@ -164,7 +168,7 @@ class LLMPlanningAgent:
|
|||||||
# The gate is this call's outcome. Recording it keeps the
|
# The gate is this call's outcome. Recording it keeps the
|
||||||
# transcript renderable: providers reject a tool call with
|
# transcript renderable: providers reject a tool call with
|
||||||
# no result, and the next planning turn replays this log.
|
# no result, and the next planning turn replays this log.
|
||||||
self._store.append_event(
|
transcript.append(
|
||||||
ToolResult(
|
ToolResult(
|
||||||
call_id=event.call_id,
|
call_id=event.call_id,
|
||||||
result=(
|
result=(
|
||||||
|
|||||||
@@ -82,6 +82,7 @@ class MilestonePublisher:
|
|||||||
OPEN_PR_INTENT,
|
OPEN_PR_INTENT,
|
||||||
{"branch": branch, "milestone": milestone, "sha": sha},
|
{"branch": branch, "milestone": milestone, "sha": sha},
|
||||||
cause=f"publishing milestone {milestone}",
|
cause=f"publishing milestone {milestone}",
|
||||||
|
work_item=task_id, # owned by the work item it publishes (L1)
|
||||||
)
|
)
|
||||||
self._git.push(self._remote, branch, token=self._token)
|
self._git.push(self._remote, branch, token=self._token)
|
||||||
# Idempotent: if a PR for this branch is already open (a crash after
|
# Idempotent: if a PR for this branch is already open (a crash after
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ resolved against external truth on boot).
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import sqlite3
|
import sqlite3
|
||||||
|
import uuid
|
||||||
from collections.abc import Callable, Mapping
|
from collections.abc import Callable, Mapping
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
@@ -21,49 +22,41 @@ from typing import Any, Self
|
|||||||
from nisse_dev.canonical import (
|
from nisse_dev.canonical import (
|
||||||
CanonicalEvent,
|
CanonicalEvent,
|
||||||
StoredEvent,
|
StoredEvent,
|
||||||
|
UserMessage,
|
||||||
event_type,
|
event_type,
|
||||||
from_payload,
|
from_payload,
|
||||||
to_payload,
|
to_payload,
|
||||||
)
|
)
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
|
|
||||||
CURRENT_SCHEMA_VERSION = 3
|
CURRENT_SCHEMA_VERSION = 4
|
||||||
|
|
||||||
|
# Stores older than this predate work items (Development Plan M1.1). There was
|
||||||
|
# no production data to carry over, so there is no migration from them: start
|
||||||
|
# a fresh store (scripts/bringup.sh start sets the old one aside).
|
||||||
|
OLDEST_SUPPORTED_VERSION = 4
|
||||||
|
|
||||||
Migration = Callable[[sqlite3.Connection], None]
|
Migration = Callable[[sqlite3.Connection], None]
|
||||||
|
|
||||||
_CANONICAL_EVENTS_DDL = """
|
|
||||||
CREATE TABLE canonical_events (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
ts TEXT NOT NULL,
|
|
||||||
type TEXT NOT NULL,
|
|
||||||
payload TEXT NOT NULL
|
|
||||||
)
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
_EVAL_OUTCOMES_DDL = """
|
|
||||||
CREATE TABLE eval_outcomes (
|
|
||||||
event_id INTEGER PRIMARY KEY,
|
|
||||||
verdict TEXT NOT NULL,
|
|
||||||
note TEXT NOT NULL DEFAULT '',
|
|
||||||
marked_at TEXT NOT NULL
|
|
||||||
)
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def _migrate_v1_add_canonical_events(conn: sqlite3.Connection) -> None:
|
|
||||||
conn.execute(_CANONICAL_EVENTS_DDL)
|
|
||||||
|
|
||||||
|
|
||||||
def _migrate_v2_add_eval_outcomes(conn: sqlite3.Connection) -> None:
|
|
||||||
conn.execute(_EVAL_OUTCOMES_DDL)
|
|
||||||
|
|
||||||
|
|
||||||
# Maps version N to the migration that takes the schema from N to N+1.
|
# Maps version N to the migration that takes the schema from N to N+1.
|
||||||
MIGRATIONS: Mapping[int, Migration] = {
|
MIGRATIONS: Mapping[int, Migration] = {}
|
||||||
1: _migrate_v1_add_canonical_events,
|
|
||||||
2: _migrate_v2_add_eval_outcomes,
|
IDLE = "IDLE"
|
||||||
}
|
|
||||||
|
# Who produced an event (L7): the operator, the orchestrator's own code, or a
|
||||||
|
# model role (L9). planner and executor become driver in the new loop (M2.3).
|
||||||
|
ACTORS = frozenset(
|
||||||
|
{
|
||||||
|
"operator",
|
||||||
|
"orchestrator",
|
||||||
|
"planner",
|
||||||
|
"executor",
|
||||||
|
"driver",
|
||||||
|
"helper",
|
||||||
|
"expert",
|
||||||
|
"classifier",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class StateSchemaError(RuntimeError):
|
class StateSchemaError(RuntimeError):
|
||||||
@@ -72,12 +65,29 @@ class StateSchemaError(RuntimeError):
|
|||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class AgentState:
|
class AgentState:
|
||||||
|
"""The state machine's view of the active work item (control.py, until
|
||||||
|
Development Plan M2.6 retires it)."""
|
||||||
|
|
||||||
state: str
|
state: str
|
||||||
project: str | None
|
project: str | None
|
||||||
data: dict[str, Any]
|
data: dict[str, Any]
|
||||||
updated_at: str
|
updated_at: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WorkItem:
|
||||||
|
id: str
|
||||||
|
scope: str | None
|
||||||
|
title: str
|
||||||
|
status: str # open | closed
|
||||||
|
outcome: str | None # completed | abandoned, once closed
|
||||||
|
position: str # where the loop is for this item
|
||||||
|
data: dict[str, Any]
|
||||||
|
opened_at: str
|
||||||
|
updated_at: str
|
||||||
|
closed_at: str | None
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Intent:
|
class Intent:
|
||||||
id: int
|
id: int
|
||||||
@@ -86,16 +96,30 @@ class Intent:
|
|||||||
status: str
|
status: str
|
||||||
created_at: str
|
created_at: str
|
||||||
resolved_at: str | None
|
resolved_at: str | None
|
||||||
|
work_item: str | None = None
|
||||||
|
|
||||||
|
|
||||||
_SCHEMA = [
|
_SCHEMA = [
|
||||||
"CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)",
|
"CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)",
|
||||||
"""
|
"""
|
||||||
CREATE TABLE agent_state (
|
CREATE TABLE work_items (
|
||||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
id TEXT PRIMARY KEY,
|
||||||
state TEXT NOT NULL,
|
scope TEXT,
|
||||||
project TEXT,
|
title TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL CHECK (status IN ('open', 'closed')),
|
||||||
|
outcome TEXT,
|
||||||
|
position TEXT NOT NULL,
|
||||||
data TEXT NOT NULL DEFAULT '{}',
|
data TEXT NOT NULL DEFAULT '{}',
|
||||||
|
opened_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL,
|
||||||
|
closed_at TEXT
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE TABLE agent (
|
||||||
|
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||||
|
active_item TEXT REFERENCES work_items(id),
|
||||||
|
idle_from_event INTEGER NOT NULL DEFAULT 0,
|
||||||
updated_at TEXT NOT NULL
|
updated_at TEXT NOT NULL
|
||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
@@ -106,11 +130,29 @@ _SCHEMA = [
|
|||||||
payload TEXT NOT NULL,
|
payload TEXT NOT NULL,
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
status TEXT NOT NULL DEFAULT 'pending',
|
||||||
created_at TEXT NOT NULL,
|
created_at TEXT NOT NULL,
|
||||||
resolved_at TEXT
|
resolved_at TEXT,
|
||||||
|
work_item TEXT
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE TABLE canonical_events (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
type TEXT NOT NULL,
|
||||||
|
payload TEXT NOT NULL,
|
||||||
|
work_item TEXT,
|
||||||
|
actor TEXT NOT NULL
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"CREATE INDEX canonical_events_by_item ON canonical_events (work_item, id)",
|
||||||
|
"""
|
||||||
|
CREATE TABLE eval_outcomes (
|
||||||
|
event_id INTEGER PRIMARY KEY,
|
||||||
|
verdict TEXT NOT NULL,
|
||||||
|
note TEXT NOT NULL DEFAULT '',
|
||||||
|
marked_at TEXT NOT NULL
|
||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
_CANONICAL_EVENTS_DDL,
|
|
||||||
_EVAL_OUTCOMES_DDL,
|
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -150,6 +192,12 @@ class StateStore:
|
|||||||
self._in_transaction(self._create_schema)
|
self._in_transaction(self._create_schema)
|
||||||
return
|
return
|
||||||
stored = self.schema_version
|
stored = self.schema_version
|
||||||
|
if stored < OLDEST_SUPPORTED_VERSION <= self._target_version:
|
||||||
|
raise StateSchemaError(
|
||||||
|
f"state store schema version {stored} predates work items "
|
||||||
|
f"(version {OLDEST_SUPPORTED_VERSION}); start a fresh store "
|
||||||
|
"(scripts/bringup.sh start sets the old one aside)"
|
||||||
|
)
|
||||||
if stored > self._target_version:
|
if stored > self._target_version:
|
||||||
raise StateSchemaError(
|
raise StateSchemaError(
|
||||||
f"state schema version {stored} is newer than this code "
|
f"state schema version {stored} is newer than this code "
|
||||||
@@ -200,17 +248,177 @@ class StateStore:
|
|||||||
).fetchone()
|
).fetchone()
|
||||||
return int(row["value"])
|
return int(row["value"])
|
||||||
|
|
||||||
def get_state(self) -> AgentState | None:
|
# --- work items (L1) ---------------------------------------------------
|
||||||
row = self._conn.execute("SELECT * FROM agent_state WHERE id = 1").fetchone()
|
|
||||||
if row is None:
|
def _ensure_agent_row(self) -> None:
|
||||||
return None
|
self._conn.execute(
|
||||||
return AgentState(
|
"INSERT INTO agent (id, active_item, updated_at) VALUES (1, NULL, ?) "
|
||||||
state=row["state"],
|
"ON CONFLICT(id) DO NOTHING",
|
||||||
project=row["project"],
|
(_now(),),
|
||||||
data=json.loads(row["data"]),
|
|
||||||
updated_at=row["updated_at"],
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _item(row: sqlite3.Row) -> WorkItem:
|
||||||
|
return WorkItem(
|
||||||
|
id=row["id"],
|
||||||
|
scope=row["scope"],
|
||||||
|
title=row["title"],
|
||||||
|
status=row["status"],
|
||||||
|
outcome=row["outcome"],
|
||||||
|
position=row["position"],
|
||||||
|
data=json.loads(row["data"]),
|
||||||
|
opened_at=row["opened_at"],
|
||||||
|
updated_at=row["updated_at"],
|
||||||
|
closed_at=row["closed_at"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def work_item(self, item_id: str) -> WorkItem | None:
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT * FROM work_items WHERE id = ?", (item_id,)
|
||||||
|
).fetchone()
|
||||||
|
return None if row is None else self._item(row)
|
||||||
|
|
||||||
|
def work_items(self, status: str | None = None) -> list[WorkItem]:
|
||||||
|
query = "SELECT * FROM work_items"
|
||||||
|
args: tuple[Any, ...] = ()
|
||||||
|
if status is not None:
|
||||||
|
query += " WHERE status = ?"
|
||||||
|
args = (status,)
|
||||||
|
rows = self._conn.execute(query + " ORDER BY opened_at", args).fetchall()
|
||||||
|
return [self._item(row) for row in rows]
|
||||||
|
|
||||||
|
def active_item(self) -> WorkItem | None:
|
||||||
|
row = self._conn.execute(
|
||||||
|
"SELECT w.* FROM agent a JOIN work_items w ON w.id = a.active_item "
|
||||||
|
"WHERE a.id = 1"
|
||||||
|
).fetchone()
|
||||||
|
return None if row is None else self._item(row)
|
||||||
|
|
||||||
|
def open_work_item(
|
||||||
|
self,
|
||||||
|
item_id: str,
|
||||||
|
*,
|
||||||
|
title: str,
|
||||||
|
scope: str | None = None,
|
||||||
|
position: str,
|
||||||
|
data: dict[str, Any] | None = None,
|
||||||
|
cause: str,
|
||||||
|
) -> WorkItem:
|
||||||
|
now = _now()
|
||||||
|
self._conn.execute(
|
||||||
|
"INSERT INTO work_items (id, scope, title, status, position, data, "
|
||||||
|
"opened_at, updated_at) VALUES (?, ?, ?, 'open', ?, ?, ?, ?)",
|
||||||
|
(item_id, scope, title, position, json.dumps(data or {}), now, now),
|
||||||
|
)
|
||||||
|
self.oplog.emit("work_item.open", cause=cause, work_item=item_id, scope=scope)
|
||||||
|
item = self.work_item(item_id)
|
||||||
|
assert item is not None
|
||||||
|
return item
|
||||||
|
|
||||||
|
def set_active(self, item_id: str | None, *, cause: str) -> None:
|
||||||
|
"""Point the agent at a work item, or at none (idle). Going idle
|
||||||
|
records where in the log idleness began, so messages sent while idle
|
||||||
|
can be carried into the next work item (carry_idle_messages)."""
|
||||||
|
self._ensure_agent_row()
|
||||||
|
previous = self._conn.execute(
|
||||||
|
"SELECT active_item FROM agent WHERE id = 1"
|
||||||
|
).fetchone()["active_item"]
|
||||||
|
if item_id is None:
|
||||||
|
last = self._conn.execute(
|
||||||
|
"SELECT COALESCE(MAX(id), 0) AS last FROM canonical_events"
|
||||||
|
).fetchone()["last"]
|
||||||
|
self._conn.execute(
|
||||||
|
"UPDATE agent SET active_item = NULL, idle_from_event = ?, "
|
||||||
|
"updated_at = ? WHERE id = 1",
|
||||||
|
(last, _now()),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
self._conn.execute(
|
||||||
|
"UPDATE agent SET active_item = ?, updated_at = ? WHERE id = 1",
|
||||||
|
(item_id, _now()),
|
||||||
|
)
|
||||||
|
if previous != item_id:
|
||||||
|
self.oplog.emit(
|
||||||
|
"work_item.switch",
|
||||||
|
cause=cause,
|
||||||
|
**{"from": previous, "to": item_id},
|
||||||
|
)
|
||||||
|
|
||||||
|
def update_work_item(
|
||||||
|
self,
|
||||||
|
item_id: str,
|
||||||
|
*,
|
||||||
|
position: str,
|
||||||
|
data: dict[str, Any],
|
||||||
|
cause: str,
|
||||||
|
) -> None:
|
||||||
|
"""Record a work item's new position and data; the full data is
|
||||||
|
passed every time (no partial updates)."""
|
||||||
|
previous = self.work_item(item_id)
|
||||||
|
if previous is None:
|
||||||
|
raise KeyError(f"no work item {item_id!r}")
|
||||||
|
self._conn.execute(
|
||||||
|
"UPDATE work_items SET position = ?, data = ?, updated_at = ? WHERE id = ?",
|
||||||
|
(position, json.dumps(data), _now(), item_id),
|
||||||
|
)
|
||||||
|
self.oplog.emit(
|
||||||
|
"state.transition",
|
||||||
|
cause=cause,
|
||||||
|
**{"from": previous.position, "to": position, "project": previous.scope},
|
||||||
|
)
|
||||||
|
|
||||||
|
def close_work_item(self, item_id: str, *, outcome: str, cause: str) -> None:
|
||||||
|
now = _now()
|
||||||
|
row = self._conn.execute(
|
||||||
|
"UPDATE work_items SET status = 'closed', outcome = ?, closed_at = ?, "
|
||||||
|
"updated_at = ? WHERE id = ? AND status = 'open' RETURNING id",
|
||||||
|
(outcome, now, now, item_id),
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
raise KeyError(f"no open work item {item_id!r}")
|
||||||
|
self.oplog.emit(
|
||||||
|
"work_item.close", cause=cause, work_item=item_id, outcome=outcome
|
||||||
|
)
|
||||||
|
|
||||||
|
def carry_idle_messages(self, item_id: str) -> int:
|
||||||
|
"""Copy what the operator said while the agent was idle into a work
|
||||||
|
item that has just opened (D1: messages sent before `use` are kept
|
||||||
|
for it). Copied as new events, with the original's ID in the source,
|
||||||
|
never by reassigning the originals: the log is never edited (L7)."""
|
||||||
|
self._ensure_agent_row()
|
||||||
|
since = self._conn.execute(
|
||||||
|
"SELECT idle_from_event FROM agent WHERE id = 1"
|
||||||
|
).fetchone()["idle_from_event"]
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT * FROM canonical_events WHERE work_item IS NULL "
|
||||||
|
"AND actor = 'operator' AND type = 'user_message' AND id > ? ORDER BY id",
|
||||||
|
(since,),
|
||||||
|
).fetchall()
|
||||||
|
log = self.transcript(item_id, "operator")
|
||||||
|
for row in rows:
|
||||||
|
payload = json.loads(row["payload"])
|
||||||
|
log.append(
|
||||||
|
UserMessage(
|
||||||
|
text=payload["text"],
|
||||||
|
source=f"{payload.get('source', '')} (sent while idle, #{row['id']})",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return len(rows)
|
||||||
|
|
||||||
|
# --- the state machine's view (control.py, until M2.6) -----------------
|
||||||
|
|
||||||
|
def get_state(self) -> AgentState | None:
|
||||||
|
"""The active work item as the state machine sees it: its position
|
||||||
|
is the state, IDLE when there is none. None only for a store the
|
||||||
|
agent has never started on."""
|
||||||
|
agent = self._conn.execute("SELECT * FROM agent WHERE id = 1").fetchone()
|
||||||
|
if agent is None:
|
||||||
|
return None
|
||||||
|
item = self.active_item()
|
||||||
|
if item is None:
|
||||||
|
return AgentState(IDLE, None, {}, agent["updated_at"])
|
||||||
|
return AgentState(item.position, item.scope, item.data, item.updated_at)
|
||||||
|
|
||||||
def transition(
|
def transition(
|
||||||
self,
|
self,
|
||||||
state: str,
|
state: str,
|
||||||
@@ -219,36 +427,55 @@ class StateStore:
|
|||||||
project: str | None = None,
|
project: str | None = None,
|
||||||
data: dict[str, Any] | None = None,
|
data: dict[str, Any] | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Record the new state-machine position; the full position is passed
|
"""The state machine's transition, over work items: IDLE closes and
|
||||||
every time (no partial updates), committed and fsynced before return."""
|
deactivates the active item (its outcome should already be recorded
|
||||||
previous = self.get_state()
|
with close_work_item); any other state updates the active item,
|
||||||
self._conn.execute(
|
opening one if there is none."""
|
||||||
"""
|
self._ensure_agent_row()
|
||||||
INSERT INTO agent_state (id, state, project, data, updated_at)
|
active = self.active_item()
|
||||||
VALUES (1, ?, ?, ?, ?)
|
if state == IDLE:
|
||||||
ON CONFLICT(id) DO UPDATE SET
|
if active is not None:
|
||||||
state = excluded.state,
|
if active.status == "open":
|
||||||
project = excluded.project,
|
self.close_work_item(active.id, outcome="closed", cause=cause)
|
||||||
data = excluded.data,
|
self.set_active(None, cause=cause)
|
||||||
updated_at = excluded.updated_at
|
self.oplog.emit(
|
||||||
""",
|
"state.transition",
|
||||||
(state, project, json.dumps(data or {}), _now()),
|
cause=cause,
|
||||||
)
|
**{
|
||||||
self.oplog.emit(
|
"from": active.position if active else None,
|
||||||
"state.transition",
|
"to": IDLE,
|
||||||
cause=cause,
|
"project": project,
|
||||||
**{
|
},
|
||||||
"from": previous.state if previous else None,
|
)
|
||||||
"to": state,
|
return
|
||||||
"project": project,
|
if active is None:
|
||||||
},
|
item_id = str((data or {}).get("task_id") or f"task-{uuid.uuid4().hex[:8]}")
|
||||||
)
|
active = self.open_work_item(
|
||||||
|
item_id,
|
||||||
|
title=project or item_id,
|
||||||
|
scope=project,
|
||||||
|
position=IDLE,
|
||||||
|
cause=cause,
|
||||||
|
)
|
||||||
|
self.set_active(item_id, cause=cause)
|
||||||
|
self.update_work_item(active.id, position=state, data=data or {}, cause=cause)
|
||||||
|
|
||||||
def begin_intent(self, kind: str, payload: dict[str, Any], *, cause: str) -> int:
|
# --- intents (L8) ------------------------------------------------------
|
||||||
"""Durably record that a unit of work is about to start."""
|
|
||||||
|
def begin_intent(
|
||||||
|
self,
|
||||||
|
kind: str,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
*,
|
||||||
|
cause: str,
|
||||||
|
work_item: str | None = None,
|
||||||
|
) -> int:
|
||||||
|
"""Durably record that a unit of work is about to start, owned by
|
||||||
|
the work item it belongs to (L1)."""
|
||||||
cursor = self._conn.execute(
|
cursor = self._conn.execute(
|
||||||
"INSERT INTO intent_log (kind, payload, created_at) VALUES (?, ?, ?)",
|
"INSERT INTO intent_log (kind, payload, created_at, work_item) "
|
||||||
(kind, json.dumps(payload), _now()),
|
"VALUES (?, ?, ?, ?)",
|
||||||
|
(kind, json.dumps(payload), _now(), work_item),
|
||||||
)
|
)
|
||||||
assert cursor.lastrowid is not None
|
assert cursor.lastrowid is not None
|
||||||
self.oplog.emit(
|
self.oplog.emit(
|
||||||
@@ -286,32 +513,51 @@ class StateStore:
|
|||||||
status=row["status"],
|
status=row["status"],
|
||||||
created_at=row["created_at"],
|
created_at=row["created_at"],
|
||||||
resolved_at=row["resolved_at"],
|
resolved_at=row["resolved_at"],
|
||||||
|
work_item=row["work_item"],
|
||||||
)
|
)
|
||||||
for row in rows
|
for row in rows
|
||||||
]
|
]
|
||||||
|
|
||||||
def append_event(self, event: CanonicalEvent) -> int:
|
# --- the canonical log (L7) -------------------------------------------
|
||||||
"""Append to the canonical transcript (L7); committed and fsynced
|
|
||||||
before return, like every other durable write."""
|
def transcript(self, work_item: str | None, actor: str) -> "Transcript":
|
||||||
|
"""The log as one role sees it for one work item: what it reads is
|
||||||
|
that item's events only, and what it writes carries the item and the
|
||||||
|
actor. None is the agent-level scope (L6)."""
|
||||||
|
if actor not in ACTORS:
|
||||||
|
raise ValueError(f"unknown actor {actor!r}")
|
||||||
|
return Transcript(self, work_item, actor)
|
||||||
|
|
||||||
|
def all_events(self, after: int = 0) -> list[StoredEvent]:
|
||||||
|
"""Every event across work items, for the few readers whose job
|
||||||
|
spans them (evaluation capture, dumps). Model calls never use this."""
|
||||||
|
rows = self._conn.execute(
|
||||||
|
"SELECT * FROM canonical_events WHERE id > ? ORDER BY id", (after,)
|
||||||
|
).fetchall()
|
||||||
|
return [_stored(row) for row in rows]
|
||||||
|
|
||||||
|
def _append(self, event: CanonicalEvent, work_item: str | None, actor: str) -> int:
|
||||||
cursor = self._conn.execute(
|
cursor = self._conn.execute(
|
||||||
"INSERT INTO canonical_events (ts, type, payload) VALUES (?, ?, ?)",
|
"INSERT INTO canonical_events (ts, type, payload, work_item, actor) "
|
||||||
(_now(), event_type(event), json.dumps(to_payload(event))),
|
"VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(
|
||||||
|
_now(),
|
||||||
|
event_type(event),
|
||||||
|
json.dumps(to_payload(event)),
|
||||||
|
work_item,
|
||||||
|
actor,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
assert cursor.lastrowid is not None
|
assert cursor.lastrowid is not None
|
||||||
return cursor.lastrowid
|
return cursor.lastrowid
|
||||||
|
|
||||||
def read_events(self, after: int = 0) -> list[StoredEvent]:
|
def _read(self, work_item: str | None, after: int) -> list[StoredEvent]:
|
||||||
rows = self._conn.execute(
|
rows = self._conn.execute(
|
||||||
"SELECT * FROM canonical_events WHERE id > ? ORDER BY id", (after,)
|
"SELECT * FROM canonical_events WHERE work_item IS ? AND id > ? "
|
||||||
|
"ORDER BY id",
|
||||||
|
(work_item, after),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return [
|
return [_stored(row) for row in rows]
|
||||||
StoredEvent(
|
|
||||||
id=row["id"],
|
|
||||||
ts=row["ts"],
|
|
||||||
event=from_payload(row["type"], json.loads(row["payload"])),
|
|
||||||
)
|
|
||||||
for row in rows
|
|
||||||
]
|
|
||||||
|
|
||||||
def mark_outcome(self, event_id: int, verdict: str, note: str = "") -> None:
|
def mark_outcome(self, event_id: int, verdict: str, note: str = "") -> None:
|
||||||
"""Record whether a delegation or consult was good (L9).
|
"""Record whether a delegation or consult was good (L9).
|
||||||
@@ -343,3 +589,45 @@ class StateStore:
|
|||||||
|
|
||||||
def __exit__(self, *exc_info: object) -> None:
|
def __exit__(self, *exc_info: object) -> None:
|
||||||
self.close()
|
self.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _stored(row: sqlite3.Row) -> StoredEvent:
|
||||||
|
return StoredEvent(
|
||||||
|
id=row["id"],
|
||||||
|
ts=row["ts"],
|
||||||
|
event=from_payload(row["type"], json.loads(row["payload"])),
|
||||||
|
work_item=row["work_item"],
|
||||||
|
actor=row["actor"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Transcript:
|
||||||
|
"""One work item's log, as one actor reads and writes it (L6, L7).
|
||||||
|
|
||||||
|
Handed to each component per call by the orchestrator, which knows the
|
||||||
|
active work item, so a component can neither read another item's events
|
||||||
|
nor write without attribution.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, store: StateStore, work_item: str | None, actor: str) -> None:
|
||||||
|
self._store = store
|
||||||
|
self.work_item = work_item
|
||||||
|
self.actor = actor
|
||||||
|
|
||||||
|
def append(self, event: CanonicalEvent) -> int:
|
||||||
|
"""Committed and fsynced before return, like every durable write."""
|
||||||
|
return self._store._append(event, self.work_item, self.actor)
|
||||||
|
|
||||||
|
def read(self, after: int = 0) -> list[StoredEvent]:
|
||||||
|
return self._store._read(self.work_item, after)
|
||||||
|
|
||||||
|
def as_actor(self, actor: str) -> "Transcript":
|
||||||
|
"""The same work item, written by another role: e.g. the executor
|
||||||
|
handing a sub-task to the helper."""
|
||||||
|
return self._store.transcript(self.work_item, actor)
|
||||||
|
|
||||||
|
def begin_intent(self, kind: str, payload: dict[str, Any], *, cause: str) -> int:
|
||||||
|
"""An intent owned by this transcript's work item."""
|
||||||
|
return self._store.begin_intent(
|
||||||
|
kind, payload, cause=cause, work_item=self.work_item
|
||||||
|
)
|
||||||
@@ -30,6 +30,7 @@ class Planner:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: object,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -64,7 +65,7 @@ class Board:
|
|||||||
|
|
||||||
|
|
||||||
class Executor:
|
class Executor:
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
return ExecutorOutcome("completed", "done")
|
return ExecutorOutcome("completed", "done")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -109,7 +109,7 @@ def test_stale_feedback_is_recorded_but_rejects_nothing(tmp_path: Path) -> None:
|
|||||||
assert gate(store)["rev"] == 2
|
assert gate(store)["rev"] == 2
|
||||||
assert "not taken as feedback" in transport.outbound[-1].text
|
assert "not taken as feedback" in transport.outbound[-1].text
|
||||||
said = [
|
said = [
|
||||||
e.event.text for e in store.read_events() if isinstance(e.event, UserMessage)
|
e.event.text for e in store.all_events() if isinstance(e.event, UserMessage)
|
||||||
]
|
]
|
||||||
assert "also, keep it in one file" in said # the planner still hears it
|
assert "also, keep it in one file" in said # the planner still hears it
|
||||||
|
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ def build(
|
|||||||
orch = Orchestrator(
|
orch = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(ScriptedClient(), store, ROLES),
|
LLMPlanningAgent(ScriptedClient(), ROLES),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=ROLES,
|
roles=ROLES,
|
||||||
gitea_operators={"dongar"},
|
gitea_operators={"dongar"},
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ from nisse_dev.canonical import (
|
|||||||
from_payload,
|
from_payload,
|
||||||
mint_call_id,
|
mint_call_id,
|
||||||
)
|
)
|
||||||
from nisse_dev.state import CURRENT_SCHEMA_VERSION, StateStore
|
from nisse_dev.state import StateSchemaError, StateStore
|
||||||
|
|
||||||
ALL_EVENTS = [
|
ALL_EVENTS = [
|
||||||
UserMessage(text="use demo", source="matrix:$abc"),
|
UserMessage(text="use demo", source="matrix:$abc"),
|
||||||
@@ -45,18 +45,20 @@ ALL_EVENTS = [
|
|||||||
def test_all_event_types_round_trip(tmp_path: Path) -> None:
|
def test_all_event_types_round_trip(tmp_path: Path) -> None:
|
||||||
with StateStore(tmp_path / "state.db") as store:
|
with StateStore(tmp_path / "state.db") as store:
|
||||||
for event in ALL_EVENTS:
|
for event in ALL_EVENTS:
|
||||||
store.append_event(event)
|
store.transcript("t", "executor").append(event)
|
||||||
with StateStore(tmp_path / "state.db") as store:
|
with StateStore(tmp_path / "state.db") as store:
|
||||||
stored = store.read_events()
|
stored = store.transcript("t", "executor").read()
|
||||||
assert [s.event for s in stored] == ALL_EVENTS
|
assert [s.event for s in stored] == ALL_EVENTS
|
||||||
assert [s.id for s in stored] == sorted(s.id for s in stored)
|
assert [s.id for s in stored] == sorted(s.id for s in stored)
|
||||||
|
assert {(s.work_item, s.actor) for s in stored} == {("t", "executor")}
|
||||||
|
|
||||||
|
|
||||||
def test_read_events_after_id(tmp_path: Path) -> None:
|
def test_read_events_after_id(tmp_path: Path) -> None:
|
||||||
with StateStore(tmp_path / "state.db") as store:
|
with StateStore(tmp_path / "state.db") as store:
|
||||||
first = store.append_event(UserMessage(text="one", source="m:1"))
|
log = store.transcript("t", "operator")
|
||||||
store.append_event(UserMessage(text="two", source="m:2"))
|
first = log.append(UserMessage(text="one", source="m:1"))
|
||||||
tail = store.read_events(after=first)
|
log.append(UserMessage(text="two", source="m:2"))
|
||||||
|
tail = log.read(after=first)
|
||||||
assert [s.event.text for s in tail] == ["two"]
|
assert [s.event.text for s in tail] == ["two"]
|
||||||
|
|
||||||
|
|
||||||
@@ -102,26 +104,17 @@ V1_SCHEMA = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def test_default_migration_upgrades_a_real_v1_store(tmp_path: Path) -> None:
|
def test_a_store_from_before_work_items_is_refused_clearly(tmp_path: Path) -> None:
|
||||||
|
# No production data predates work items, so there is no migration from
|
||||||
|
# these versions (Development Plan M1.1): the store must say so rather
|
||||||
|
# than open half-understood.
|
||||||
db = tmp_path / "state.db"
|
db = tmp_path / "state.db"
|
||||||
conn = sqlite3.connect(db)
|
conn = sqlite3.connect(db)
|
||||||
for statement in V1_SCHEMA:
|
for statement in V1_SCHEMA:
|
||||||
conn.execute(statement)
|
conn.execute(statement)
|
||||||
conn.execute("INSERT INTO meta (key, value) VALUES ('schema_version', '1')")
|
conn.execute("INSERT INTO meta (key, value) VALUES ('schema_version', '3')")
|
||||||
conn.execute(
|
|
||||||
"INSERT INTO agent_state (id, state, project, data, updated_at) "
|
|
||||||
"VALUES (1, 'IDLE', NULL, '{}', 't')"
|
|
||||||
)
|
|
||||||
conn.commit()
|
conn.commit()
|
||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
with StateStore(db) as store:
|
with pytest.raises(StateSchemaError, match="predates work items"):
|
||||||
assert store.schema_version == CURRENT_SCHEMA_VERSION
|
StateStore(db)
|
||||||
state = store.get_state()
|
|
||||||
assert state is not None
|
|
||||||
assert state.state == "IDLE" # v1 data intact
|
|
||||||
store.append_event(UserMessage(text="hello", source="m:1"))
|
|
||||||
assert len(store.read_events()) == 1
|
|
||||||
# Two migrations were traversed, so both later tables must exist.
|
|
||||||
store.mark_outcome(1, "good", "arrived via v1 -> v3")
|
|
||||||
assert store.outcomes()[1] == ("good", "arrived via v1 -> v3")
|
|
||||||
@@ -248,7 +248,7 @@ def test_the_executor_sees_only_its_milestone(tmp_path: Path) -> None:
|
|||||||
GateOpened(gate="milestone-1", reason="plan submitted"),
|
GateOpened(gate="milestone-1", reason="plan submitted"),
|
||||||
UserMessage(text="start milestone 1", source="t"),
|
UserMessage(text="start milestone 1", source="t"),
|
||||||
):
|
):
|
||||||
store.append_event(event)
|
store.transcript("t", "executor").append(event)
|
||||||
|
|
||||||
class Workspace:
|
class Workspace:
|
||||||
def describe(self) -> str:
|
def describe(self) -> str:
|
||||||
@@ -263,7 +263,7 @@ def test_the_executor_sees_only_its_milestone(tmp_path: Path) -> None:
|
|||||||
return [AssistantText(text="done", model=model)]
|
return [AssistantText(text="done", model=model)]
|
||||||
|
|
||||||
roles = RoleMap(executor="m", planner="m", helper="m", expert="m")
|
roles = RoleMap(executor="m", planner="m", helper="m", expert="m")
|
||||||
LLMExecutor(Client(), store, roles, Workspace()).run(1)
|
LLMExecutor(Client(), roles, Workspace()).run(store.transcript("t", "executor"), 1)
|
||||||
|
|
||||||
texts = [getattr(e, "text", None) for e in seen["events"]]
|
texts = [getattr(e, "text", None) for e in seen["events"]]
|
||||||
assert "start milestone 1" in texts
|
assert "start milestone 1" in texts
|
||||||
@@ -278,7 +278,9 @@ def test_a_request_that_cannot_fit_stops_the_milestone(tmp_path: Path) -> None:
|
|||||||
# Never sent: the executor stops and asks the operator, like any failed
|
# Never sent: the executor stops and asks the operator, like any failed
|
||||||
# call, rather than handing a server a prompt it would silently cut.
|
# call, rather than handing a server a prompt it would silently cut.
|
||||||
store = StateStore(tmp_path / "state.db", oplog=OpLog("t", io.StringIO()))
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("t", io.StringIO()))
|
||||||
store.append_event(UserMessage(text="start milestone 1", source="t"))
|
store.transcript("t", "operator").append(
|
||||||
|
UserMessage(text="start milestone 1", source="t")
|
||||||
|
)
|
||||||
|
|
||||||
class Workspace:
|
class Workspace:
|
||||||
def describe(self) -> str:
|
def describe(self) -> str:
|
||||||
@@ -287,7 +289,9 @@ def test_a_request_that_cannot_fit_stops_the_milestone(tmp_path: Path) -> None:
|
|||||||
inner = RecordingClient()
|
inner = RecordingClient()
|
||||||
client = BudgetedClient(inner, lambda m: (4096, 1024))
|
client = BudgetedClient(inner, lambda m: (4096, 1024))
|
||||||
roles = RoleMap(executor="m", planner="m", helper="m", expert="m")
|
roles = RoleMap(executor="m", planner="m", helper="m", expert="m")
|
||||||
outcome = LLMExecutor(client, store, roles, Workspace()).run(1)
|
outcome = LLMExecutor(client, roles, Workspace()).run(
|
||||||
|
store.transcript("t", "executor"), 1
|
||||||
|
)
|
||||||
|
|
||||||
assert outcome.status == "escalate"
|
assert outcome.status == "escalate"
|
||||||
assert "fit" in outcome.note
|
assert "fit" in outcome.note
|
||||||
|
|||||||
@@ -142,7 +142,7 @@ def test_a_message_is_matched_as_stripped_but_recorded_as_sent(
|
|||||||
h.transport.receive("don", "approve", original="Nisse Dev: approve")
|
h.transport.receive("don", "approve", original="Nisse Dev: approve")
|
||||||
assert h.state == AWAIT_MILESTONE_SIGNOFF
|
assert h.state == AWAIT_MILESTONE_SIGNOFF
|
||||||
recorded = [
|
recorded = [
|
||||||
e.event.text for e in h.store.read_events() if isinstance(e.event, UserMessage)
|
e.event.text for e in h.store.all_events() if isinstance(e.event, UserMessage)
|
||||||
]
|
]
|
||||||
assert "Nisse Dev: approve" in recorded
|
assert "Nisse Dev: approve" in recorded
|
||||||
assert "approve" not in recorded
|
assert "approve" not in recorded
|
||||||
@@ -160,6 +160,7 @@ def test_status_reports_state_and_gate(h: Harness) -> None:
|
|||||||
class QuestionThenPlan:
|
class QuestionThenPlan:
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: object,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -192,6 +193,7 @@ class FailsOnce:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: object,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
|
|||||||
@@ -19,12 +19,13 @@ from nisse_dev.delegation import (
|
|||||||
DELEGATION_INTENT,
|
DELEGATION_INTENT,
|
||||||
Caps,
|
Caps,
|
||||||
Delegation,
|
Delegation,
|
||||||
|
DelegationResult,
|
||||||
delegation_reconciler,
|
delegation_reconciler,
|
||||||
)
|
)
|
||||||
from nisse_dev.git import GitRepo
|
from nisse_dev.git import GitRepo
|
||||||
from nisse_dev.llm import BudgetExhausted, LLMError, UpstreamUnavailable
|
from nisse_dev.llm import BudgetExhausted, LLMError, UpstreamUnavailable
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
from nisse_dev.tools import Workspace
|
from nisse_dev.tools import Workspace
|
||||||
|
|
||||||
|
|
||||||
@@ -86,8 +87,34 @@ def ws(repo: GitRepo) -> Workspace:
|
|||||||
return Workspace(repo.root, timeout_s=10)
|
return Workspace(repo.root, timeout_s=10)
|
||||||
|
|
||||||
|
|
||||||
|
ITEM = "t" # the work item these delegations run in
|
||||||
|
|
||||||
|
|
||||||
|
def log(store: StateStore) -> Transcript:
|
||||||
|
return store.transcript(ITEM, "helper")
|
||||||
|
|
||||||
|
|
||||||
|
class Bound:
|
||||||
|
"""A delegation run in the test's work item, as the executor runs it."""
|
||||||
|
|
||||||
|
def __init__(self, delegation: Delegation, store: StateStore) -> None:
|
||||||
|
self.delegation = delegation
|
||||||
|
self.store = store
|
||||||
|
|
||||||
|
def run(self, **kw: Any) -> DelegationResult:
|
||||||
|
return self.delegation.run(log(self.store), **kw)
|
||||||
|
|
||||||
|
def resume(self, intent: Any) -> DelegationResult:
|
||||||
|
return self.delegation.resume(intent)
|
||||||
|
|
||||||
|
def recover(self, notify: Any) -> DelegationResult | None:
|
||||||
|
return self.delegation.recover(notify)
|
||||||
|
|
||||||
|
|
||||||
def build(client: Any, store: StateStore, ws: Workspace, repo: GitRepo, **kw: Any):
|
def build(client: Any, store: StateStore, ws: Workspace, repo: GitRepo, **kw: Any):
|
||||||
return Delegation(client, store, ws, repo, model="claude-haiku-4-5", **kw)
|
return Bound(
|
||||||
|
Delegation(client, store, ws, repo, model="claude-haiku-4-5", **kw), store
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_successful_delegation_returns_a_structured_handback(
|
def test_successful_delegation_returns_a_structured_handback(
|
||||||
@@ -169,7 +196,7 @@ def test_stop_discards_the_branch_and_settles_the_intent(
|
|||||||
# The unrun call still has a result, so the transcript renders.
|
# The unrun call still has a result, so the transcript renders.
|
||||||
skipped = [
|
skipped = [
|
||||||
e.event
|
e.event
|
||||||
for e in store.read_events()
|
for e in log(store).read()
|
||||||
if isinstance(e.event, ToolResult) and e.event.result == STOPPED_RESULT
|
if isinstance(e.event, ToolResult) and e.event.result == STOPPED_RESULT
|
||||||
]
|
]
|
||||||
assert len(skipped) == 1
|
assert len(skipped) == 1
|
||||||
@@ -225,7 +252,7 @@ def test_events_are_checkpointed_as_they_arrive(
|
|||||||
)
|
)
|
||||||
build(client, store, ws, repo).run(brief="add a", branch="d")
|
build(client, store, ws, repo).run(brief="add a", branch="d")
|
||||||
|
|
||||||
kinds = [type(e.event).__name__ for e in store.read_events()]
|
kinds = [type(e.event).__name__ for e in log(store).read()]
|
||||||
assert kinds == [
|
assert kinds == [
|
||||||
"DelegationStarted",
|
"DelegationStarted",
|
||||||
"ToolCall",
|
"ToolCall",
|
||||||
@@ -245,7 +272,7 @@ def test_composite_events_record_the_brief_caps_and_outcome(
|
|||||||
caps = Caps(turns=7, tokens=1234, wall_clock_s=42.0)
|
caps = Caps(turns=7, tokens=1234, wall_clock_s=42.0)
|
||||||
build(client, store, ws, repo, caps=caps).run(brief="the brief", branch="d")
|
build(client, store, ws, repo, caps=caps).run(brief="the brief", branch="d")
|
||||||
|
|
||||||
events = [e.event for e in store.read_events()]
|
events = [e.event for e in log(store).read()]
|
||||||
started = next(e for e in events if isinstance(e, DelegationStarted))
|
started = next(e for e in events if isinstance(e, DelegationStarted))
|
||||||
completed = next(e for e in events if isinstance(e, DelegationCompleted))
|
completed = next(e for e in events if isinstance(e, DelegationCompleted))
|
||||||
assert started.brief == "the brief"
|
assert started.brief == "the brief"
|
||||||
@@ -356,6 +383,7 @@ def pending_delegation_intent(store: StateStore, branch: str = "d") -> Any:
|
|||||||
DELEGATION_INTENT,
|
DELEGATION_INTENT,
|
||||||
{"branch": branch, "base": "main", "brief": "finish it"},
|
{"branch": branch, "base": "main", "brief": "finish it"},
|
||||||
cause="test",
|
cause="test",
|
||||||
|
work_item=ITEM, # owned by the work item it ran in
|
||||||
)
|
)
|
||||||
return store.pending_intents()[-1]
|
return store.pending_intents()[-1]
|
||||||
|
|
||||||
@@ -385,9 +413,9 @@ def test_clean_cut_after_a_tool_result_is_resumable(
|
|||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
(repo.root / "partial.py").write_text("x")
|
(repo.root / "partial.py").write_text("x")
|
||||||
repo.commit_all("partial work")
|
repo.commit_all("partial work")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolCall(call_id="c", tool="write_file", arguments={}))
|
log(store).append(ToolCall(call_id="c", tool="write_file", arguments={}))
|
||||||
store.append_event(ToolResult(call_id="c", result="wrote", success=True))
|
log(store).append(ToolResult(call_id="c", result="wrote", success=True))
|
||||||
intent = pending_delegation_intent(store)
|
intent = pending_delegation_intent(store)
|
||||||
|
|
||||||
assert delegation_reconciler(store, repo)(intent) is True
|
assert delegation_reconciler(store, repo)(intent) is True
|
||||||
@@ -398,8 +426,8 @@ def test_dangling_tool_call_gates(
|
|||||||
) -> None:
|
) -> None:
|
||||||
# The effect of the call is unknown and not verifiable by inspection.
|
# The effect of the call is unknown and not verifiable by inspection.
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolCall(call_id="c", tool="run_command", arguments={}))
|
log(store).append(ToolCall(call_id="c", tool="run_command", arguments={}))
|
||||||
intent = pending_delegation_intent(store)
|
intent = pending_delegation_intent(store)
|
||||||
|
|
||||||
assert delegation_reconciler(store, repo)(intent) is None
|
assert delegation_reconciler(store, repo)(intent) is None
|
||||||
@@ -408,8 +436,8 @@ def test_dangling_tool_call_gates(
|
|||||||
def test_missing_branch_means_nothing_started(
|
def test_missing_branch_means_nothing_started(
|
||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolResult(call_id="c", result="ok", success=True))
|
log(store).append(ToolResult(call_id="c", result="ok", success=True))
|
||||||
intent = pending_delegation_intent(store, branch="never-created")
|
intent = pending_delegation_intent(store, branch="never-created")
|
||||||
|
|
||||||
assert delegation_reconciler(store, repo)(intent) is False
|
assert delegation_reconciler(store, repo)(intent) is False
|
||||||
@@ -419,8 +447,8 @@ def test_a_finished_delegation_settles_rather_than_resumes(
|
|||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(
|
log(store).append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback="done",
|
handback="done",
|
||||||
artifacts=[],
|
artifacts=[],
|
||||||
@@ -440,8 +468,8 @@ def test_resume_continues_the_same_delegation(
|
|||||||
# Interrupt after one turn, then resume: no second delegation_started,
|
# Interrupt after one turn, then resume: no second delegation_started,
|
||||||
# because L7 defines a session as the delegation, not the process.
|
# because L7 defines a session as the delegation, not the process.
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolResult(call_id="c", result="wrote a.py", success=True))
|
log(store).append(ToolResult(call_id="c", result="wrote a.py", success=True))
|
||||||
intent = pending_delegation_intent(store)
|
intent = pending_delegation_intent(store)
|
||||||
|
|
||||||
client = ScriptedClient(
|
client = ScriptedClient(
|
||||||
@@ -450,7 +478,7 @@ def test_resume_continues_the_same_delegation(
|
|||||||
result = build(client, store, ws, repo).resume(intent)
|
result = build(client, store, ws, repo).resume(intent)
|
||||||
|
|
||||||
assert result.usable
|
assert result.usable
|
||||||
starts = [e for e in store.read_events() if isinstance(e.event, DelegationStarted)]
|
starts = [e for e in log(store).read() if isinstance(e.event, DelegationStarted)]
|
||||||
assert len(starts) == 1
|
assert len(starts) == 1
|
||||||
assert store.pending_intents() == []
|
assert store.pending_intents() == []
|
||||||
assert (repo.root / "b.py").exists()
|
assert (repo.root / "b.py").exists()
|
||||||
@@ -474,8 +502,8 @@ def test_recover_resumes_a_clean_cut_with_a_notice(
|
|||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolResult(call_id="c", result="wrote a.py", success=True))
|
log(store).append(ToolResult(call_id="c", result="wrote a.py", success=True))
|
||||||
pending_delegation_intent(store)
|
pending_delegation_intent(store)
|
||||||
notices: list[str] = []
|
notices: list[str] = []
|
||||||
|
|
||||||
@@ -493,8 +521,8 @@ def test_recover_settles_a_finished_delegation_without_resuming(
|
|||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(
|
log(store).append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback="done",
|
handback="done",
|
||||||
artifacts=[],
|
artifacts=[],
|
||||||
@@ -514,7 +542,7 @@ def test_recover_settles_a_finished_delegation_without_resuming(
|
|||||||
def test_recover_discards_an_intent_whose_branch_never_existed(
|
def test_recover_discards_an_intent_whose_branch_never_existed(
|
||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
store.append_event(ToolResult(call_id="c", result="ok", success=True))
|
log(store).append(ToolResult(call_id="c", result="ok", success=True))
|
||||||
pending_delegation_intent(store, branch="never-made")
|
pending_delegation_intent(store, branch="never-made")
|
||||||
|
|
||||||
assert build(ScriptedClient([]), store, ws, repo).recover(lambda _m: None) is None
|
assert build(ScriptedClient([]), store, ws, repo).recover(lambda _m: None) is None
|
||||||
@@ -525,8 +553,8 @@ def test_recover_leaves_an_unclean_cut_for_the_boot_gate(
|
|||||||
store: StateStore, ws: Workspace, repo: GitRepo
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
) -> None:
|
) -> None:
|
||||||
repo.checkout_new_branch("d")
|
repo.checkout_new_branch("d")
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
log(store).append(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
||||||
store.append_event(ToolCall(call_id="c", tool="run_command", arguments={}))
|
log(store).append(ToolCall(call_id="c", tool="run_command", arguments={}))
|
||||||
pending_delegation_intent(store)
|
pending_delegation_intent(store)
|
||||||
|
|
||||||
assert build(ScriptedClient([]), store, ws, repo).recover(lambda _m: None) is None
|
assert build(ScriptedClient([]), store, ws, repo).recover(lambda _m: None) is None
|
||||||
@@ -581,7 +609,7 @@ def test_tokens_spent_are_reported_not_left_at_zero(
|
|||||||
assert result.tokens == 2000 # two calls, 1000 each
|
assert result.tokens == 2000 # two calls, 1000 each
|
||||||
|
|
||||||
completed = next(
|
completed = next(
|
||||||
e.event for e in store.read_events() if isinstance(e.event, DelegationCompleted)
|
e.event for e in log(store).read() if isinstance(e.event, DelegationCompleted)
|
||||||
)
|
)
|
||||||
# L9 evaluates cost against outcome, which needs a real number.
|
# L9 evaluates cost against outcome, which needs a real number.
|
||||||
assert completed.tokens == 2000
|
assert completed.tokens == 2000
|
||||||
|
|||||||
@@ -9,34 +9,43 @@ from nisse_dev.state import StateStore
|
|||||||
|
|
||||||
def store_with_a_run(path: Path) -> None:
|
def store_with_a_run(path: Path) -> None:
|
||||||
store = StateStore(path, oplog=OpLog("nisse-dev", io.StringIO()))
|
store = StateStore(path, oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
store.transcript(None, "operator").append(
|
||||||
|
UserMessage(text="hello before use", source="matrix")
|
||||||
|
)
|
||||||
store.transition(
|
store.transition(
|
||||||
"AWAIT_ARCH_SIGNOFF",
|
"AWAIT_ARCH_SIGNOFF",
|
||||||
cause="test",
|
cause="test",
|
||||||
project="demo",
|
project="demo",
|
||||||
data={"task_id": "demo-1", "gate": {"name": "arch", "rev": 1}},
|
data={"task_id": "demo-1", "gate": {"name": "arch", "rev": 1}},
|
||||||
)
|
)
|
||||||
store.append_event(UserMessage(text="Add a truncate helper", source="matrix"))
|
log = store.transcript("demo-1", "operator")
|
||||||
store.append_event(PlanSubmitted(plan="x" * 500, model="opus"))
|
log.append(UserMessage(text="Add a truncate helper", source="matrix"))
|
||||||
store.append_event(
|
log.as_actor("planner").append(PlanSubmitted(plan="x" * 500, model="opus"))
|
||||||
|
executor = log.as_actor("executor")
|
||||||
|
executor.append(
|
||||||
ToolCall(call_id="c1", tool="run_command", arguments={"command": "ls"})
|
ToolCall(call_id="c1", tool="run_command", arguments={"command": "ls"})
|
||||||
)
|
)
|
||||||
store.append_event(ToolResult(call_id="c1", result="README.md", success=False))
|
executor.append(ToolResult(call_id="c1", result="README.md", success=False))
|
||||||
store.begin_intent("open_pr", {"branch": "b"}, cause="test")
|
log.begin_intent("open_pr", {"branch": "b"}, cause="test")
|
||||||
store.close()
|
store.close()
|
||||||
|
|
||||||
|
|
||||||
def test_the_dump_shows_state_intents_and_each_event(tmp_path: Path) -> None:
|
def test_the_dump_shows_items_intents_and_each_event_with_its_actor(
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
db = tmp_path / "state.db"
|
db = tmp_path / "state.db"
|
||||||
store_with_a_run(db)
|
store_with_a_run(db)
|
||||||
text = render(db)
|
text = render(db)
|
||||||
|
assert "active work item: demo-1" in text
|
||||||
assert (
|
assert (
|
||||||
"state: AWAIT_ARCH_SIGNOFF project: demo task: demo-1 gate: arch rev-1"
|
"demo-1 open position: AWAIT_ARCH_SIGNOFF scope: demo gate: arch rev-1"
|
||||||
in text
|
in (text)
|
||||||
)
|
)
|
||||||
assert "pending intent #1 open_pr" in text
|
assert "pending intent #1 open_pr [demo-1]" in text
|
||||||
assert "USER [matrix]: Add a truncate helper" in text
|
assert "[agent/operator] USER [matrix]: hello before use" in text
|
||||||
assert "PLAN (opus, 500 chars): " in text and "[340 more]" in text
|
assert "[demo-1/operator] USER [matrix]: Add a truncate helper" in text
|
||||||
assert 'CALL run_command {"command": "ls"}' in text
|
assert "[demo-1/planner] PLAN (opus, 500 chars): " in text and "[340 more]" in text
|
||||||
|
assert '[demo-1/executor] CALL run_command {"command": "ls"}' in text
|
||||||
assert "result (FAILED): README.md" in text
|
assert "result (FAILED): README.md" in text
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -27,10 +27,10 @@ def record_pair(
|
|||||||
role: str = "helper",
|
role: str = "helper",
|
||||||
status: str = "handback",
|
status: str = "handback",
|
||||||
) -> int:
|
) -> int:
|
||||||
store.append_event(
|
store.transcript("t", "helper").append(
|
||||||
DelegationStarted(brief=brief, role=role, model=model, caps={"turns": 5})
|
DelegationStarted(brief=brief, role=role, model=model, caps={"turns": 5})
|
||||||
)
|
)
|
||||||
store.append_event(
|
store.transcript("t", "helper").append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback="what changed: greet.py",
|
handback="what changed: greet.py",
|
||||||
artifacts=["greet.py"],
|
artifacts=["greet.py"],
|
||||||
@@ -39,7 +39,7 @@ def record_pair(
|
|||||||
exit_status=status,
|
exit_status=status,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return store.read_events()[-1].id
|
return store.all_events()[-1].id
|
||||||
|
|
||||||
|
|
||||||
def test_pairs_a_start_with_its_completion(store: StateStore) -> None:
|
def test_pairs_a_start_with_its_completion(store: StateStore) -> None:
|
||||||
@@ -55,16 +55,18 @@ def test_pairs_a_start_with_its_completion(store: StateStore) -> None:
|
|||||||
def test_unpaired_start_is_skipped(store: StateStore) -> None:
|
def test_unpaired_start_is_skipped(store: StateStore) -> None:
|
||||||
# A crash mid-delegation leaves a start with no completion; a half-formed
|
# A crash mid-delegation leaves a start with no completion; a half-formed
|
||||||
# record would pollute the set L9 exists to make trustworthy.
|
# record would pollute the set L9 exists to make trustworthy.
|
||||||
store.append_event(
|
store.transcript("t", "helper").append(
|
||||||
DelegationStarted(brief="died", role="helper", model="m", caps={})
|
DelegationStarted(brief="died", role="helper", model="m", caps={})
|
||||||
)
|
)
|
||||||
assert collect(store) == []
|
assert collect(store) == []
|
||||||
|
|
||||||
|
|
||||||
def test_unrelated_events_between_the_pair_are_ignored(store: StateStore) -> None:
|
def test_unrelated_events_between_the_pair_are_ignored(store: StateStore) -> None:
|
||||||
store.append_event(DelegationStarted(brief="b", role="helper", model="m", caps={}))
|
store.transcript("t", "helper").append(
|
||||||
store.append_event(AssistantText(text="working", model="m"))
|
DelegationStarted(brief="b", role="helper", model="m", caps={})
|
||||||
store.append_event(
|
)
|
||||||
|
store.transcript("t", "helper").append(AssistantText(text="working", model="m"))
|
||||||
|
store.transcript("t", "helper").append(
|
||||||
DelegationCompleted(
|
DelegationCompleted(
|
||||||
handback="done",
|
handback="done",
|
||||||
artifacts=[],
|
artifacts=[],
|
||||||
@@ -103,7 +105,7 @@ def test_consults_and_delegations_share_the_set(store: StateStore) -> None:
|
|||||||
|
|
||||||
def test_collect_can_start_after_a_marker(store: StateStore) -> None:
|
def test_collect_can_start_after_a_marker(store: StateStore) -> None:
|
||||||
record_pair(store, brief="old")
|
record_pair(store, brief="old")
|
||||||
marker = store.read_events()[-1].id
|
marker = store.all_events()[-1].id
|
||||||
record_pair(store, brief="new")
|
record_pair(store, brief="new")
|
||||||
assert [r.brief for r in collect(store, after=marker)] == ["new"]
|
assert [r.brief for r in collect(store, after=marker)] == ["new"]
|
||||||
|
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ from nisse_dev.canonical import (
|
|||||||
from nisse_dev.executor import LLMExecutor
|
from nisse_dev.executor import LLMExecutor
|
||||||
from nisse_dev.llm import LLMError, RoleMap
|
from nisse_dev.llm import LLMError, RoleMap
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
from nisse_dev.tools import Workspace
|
from nisse_dev.tools import Workspace
|
||||||
|
|
||||||
ROLES = RoleMap(
|
ROLES = RoleMap(
|
||||||
@@ -66,8 +66,26 @@ def ws(tmp_path: Path) -> Workspace:
|
|||||||
return Workspace(tmp_path / "work", timeout_s=10)
|
return Workspace(tmp_path / "work", timeout_s=10)
|
||||||
|
|
||||||
|
|
||||||
def build(client: Any, store: StateStore, ws: Workspace, **kw: Any) -> LLMExecutor:
|
ITEM = "t" # the work item these tests run in
|
||||||
return LLMExecutor(client, store, ROLES, ws, **kw)
|
|
||||||
|
|
||||||
|
def log(store: StateStore) -> Transcript:
|
||||||
|
return store.transcript(ITEM, "executor")
|
||||||
|
|
||||||
|
|
||||||
|
class Bound:
|
||||||
|
"""An executor run in the test's work item, as the orchestrator runs it."""
|
||||||
|
|
||||||
|
def __init__(self, executor: LLMExecutor, store: StateStore) -> None:
|
||||||
|
self.executor = executor
|
||||||
|
self.store = store
|
||||||
|
|
||||||
|
def run(self, milestone: int) -> Any:
|
||||||
|
return self.executor.run(log(self.store), milestone)
|
||||||
|
|
||||||
|
|
||||||
|
def build(client: Any, store: StateStore, ws: Workspace, **kw: Any) -> Bound:
|
||||||
|
return Bound(LLMExecutor(client, ROLES, ws, **kw), store)
|
||||||
|
|
||||||
|
|
||||||
def test_tool_calls_are_executed_and_results_logged(
|
def test_tool_calls_are_executed_and_results_logged(
|
||||||
@@ -84,7 +102,7 @@ def test_tool_calls_are_executed_and_results_logged(
|
|||||||
assert outcome.status == "completed"
|
assert outcome.status == "completed"
|
||||||
assert outcome.note == "wrote the file"
|
assert outcome.note == "wrote the file"
|
||||||
assert (ws.root / "app.py").read_text() == "print(1)\n"
|
assert (ws.root / "app.py").read_text() == "print(1)\n"
|
||||||
kinds = [type(e.event).__name__ for e in store.read_events()]
|
kinds = [type(e.event).__name__ for e in log(store).read()]
|
||||||
assert kinds == ["ToolCall", "ToolResult", "ToolCall", "ToolResult"]
|
assert kinds == ["ToolCall", "ToolResult", "ToolCall", "ToolResult"]
|
||||||
|
|
||||||
|
|
||||||
@@ -138,7 +156,7 @@ def test_finishing_without_any_work_is_refused_once(
|
|||||||
assert (ws.root / "a.py").exists() # the refusal produced actual work
|
assert (ws.root / "a.py").exists() # the refusal produced actual work
|
||||||
[refusal] = [
|
[refusal] = [
|
||||||
e.event
|
e.event
|
||||||
for e in store.read_events()
|
for e in log(store).read()
|
||||||
if isinstance(e.event, ToolResult) and e.event.call_id == "f1"
|
if isinstance(e.event, ToolResult) and e.event.call_id == "f1"
|
||||||
]
|
]
|
||||||
assert not refusal.success
|
assert not refusal.success
|
||||||
@@ -165,7 +183,7 @@ def test_calls_after_finish_are_not_run(store: StateStore, ws: Workspace) -> Non
|
|||||||
assert not (ws.root / "b.py").exists()
|
assert not (ws.root / "b.py").exists()
|
||||||
[late] = [
|
[late] = [
|
||||||
e.event
|
e.event
|
||||||
for e in store.read_events()
|
for e in log(store).read()
|
||||||
if isinstance(e.event, ToolResult) and e.event.call_id == "late"
|
if isinstance(e.event, ToolResult) and e.event.call_id == "late"
|
||||||
]
|
]
|
||||||
assert "finish had already ended the milestone" in late.result
|
assert "finish had already ended the milestone" in late.result
|
||||||
@@ -183,7 +201,7 @@ def test_an_invented_tool_counts_toward_escalation(
|
|||||||
outcome = build(client, store, ws, failure_limit=2).run(milestone=1)
|
outcome = build(client, store, ws, failure_limit=2).run(milestone=1)
|
||||||
assert outcome.status == "escalate"
|
assert outcome.status == "escalate"
|
||||||
assert len(client.calls) == 2
|
assert len(client.calls) == 2
|
||||||
results = [e.event for e in store.read_events() if isinstance(e.event, ToolResult)]
|
results = [e.event for e in log(store).read() if isinstance(e.event, ToolResult)]
|
||||||
assert "no such tool: chat" in results[0].result
|
assert "no such tool: chat" in results[0].result
|
||||||
assert "finish" in results[0].result # it is told what does exist
|
assert "finish" in results[0].result # it is told what does exist
|
||||||
|
|
||||||
@@ -280,7 +298,7 @@ def test_multiple_calls_in_one_turn_all_run(store: StateStore, ws: Workspace) ->
|
|||||||
build(client, store, ws).run(milestone=1)
|
build(client, store, ws).run(milestone=1)
|
||||||
assert (ws.root / "one.txt").exists() and (ws.root / "two.txt").exists()
|
assert (ws.root / "one.txt").exists() and (ws.root / "two.txt").exists()
|
||||||
results = [
|
results = [
|
||||||
e.event for e in store.read_events() if type(e.event).__name__ == "ToolResult"
|
e.event for e in log(store).read() if type(e.event).__name__ == "ToolResult"
|
||||||
]
|
]
|
||||||
assert {r.call_id for r in results} == {"a", "b", "fin"}
|
assert {r.call_id for r in results} == {"a", "b", "fin"}
|
||||||
|
|
||||||
@@ -293,7 +311,7 @@ class StubDesk:
|
|||||||
self._answer = answer
|
self._answer = answer
|
||||||
self._error = error
|
self._error = error
|
||||||
|
|
||||||
def ask(self, question: str, **kw: Any) -> Any:
|
def ask(self, transcript: Any, question: str, **kw: Any) -> Any:
|
||||||
self.asked.append(question)
|
self.asked.append(question)
|
||||||
if self._error is not None:
|
if self._error is not None:
|
||||||
raise self._error
|
raise self._error
|
||||||
@@ -336,7 +354,7 @@ def test_ask_expert_answer_comes_back_as_a_tool_result(
|
|||||||
|
|
||||||
assert desk.asked == ["sqlite or postgres?"]
|
assert desk.asked == ["sqlite or postgres?"]
|
||||||
results = [
|
results = [
|
||||||
e.event for e in store.read_events() if type(e.event).__name__ == "ToolResult"
|
e.event for e in log(store).read() if type(e.event).__name__ == "ToolResult"
|
||||||
]
|
]
|
||||||
assert "prefer sqlite here" in results[0].result
|
assert "prefer sqlite here" in results[0].result
|
||||||
|
|
||||||
@@ -372,7 +390,7 @@ def test_exhausted_budget_is_reported_to_the_model_not_raised(
|
|||||||
|
|
||||||
assert outcome.status == "completed" # it got on with the work
|
assert outcome.status == "completed" # it got on with the work
|
||||||
results = [
|
results = [
|
||||||
e.event for e in store.read_events() if type(e.event).__name__ == "ToolResult"
|
e.event for e in log(store).read() if type(e.event).__name__ == "ToolResult"
|
||||||
]
|
]
|
||||||
assert not results[0].success
|
assert not results[0].success
|
||||||
assert "budget" in results[0].result
|
assert "budget" in results[0].result
|
||||||
@@ -20,7 +20,7 @@ from nisse_dev.expert import (
|
|||||||
)
|
)
|
||||||
from nisse_dev.llm import LLMError, RoleMap
|
from nisse_dev.llm import LLMError, RoleMap
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
|
|
||||||
ROLES = RoleMap(
|
ROLES = RoleMap(
|
||||||
executor="llama3.2",
|
executor="llama3.2",
|
||||||
@@ -57,8 +57,25 @@ def store(tmp_path: Path) -> StateStore:
|
|||||||
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
def desk(client: Any, store: StateStore, **kw: Any) -> ExpertDesk:
|
ITEM = "t" # the work item the consults happen in
|
||||||
return ExpertDesk(client, store, ROLES, **kw)
|
|
||||||
|
|
||||||
|
def log(store: StateStore, item: str = ITEM) -> Transcript:
|
||||||
|
return store.transcript(item, "expert")
|
||||||
|
|
||||||
|
|
||||||
|
class Bound:
|
||||||
|
"""The desk as the executor uses it: within one work item."""
|
||||||
|
|
||||||
|
def __init__(self, desk: ExpertDesk, store: StateStore, item: str) -> None:
|
||||||
|
self.desk, self.store, self.item = desk, store, item
|
||||||
|
|
||||||
|
def ask(self, question: str, **kw: Any) -> Any:
|
||||||
|
return self.desk.ask(log(self.store, self.item), question, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def desk(client: Any, store: StateStore, item: str = ITEM, **kw: Any) -> Bound:
|
||||||
|
return Bound(ExpertDesk(client, ROLES, **kw), store, item)
|
||||||
|
|
||||||
|
|
||||||
# --- parsing (the contract is prompt-level, so parsing is the guard) ------
|
# --- parsing (the contract is prompt-level, so parsing is the guard) ------
|
||||||
@@ -104,7 +121,7 @@ def test_ask_returns_a_validated_answer(store: StateStore) -> None:
|
|||||||
def test_consult_is_stateless_and_uses_the_consultant_role(
|
def test_consult_is_stateless_and_uses_the_consultant_role(
|
||||||
store: StateStore,
|
store: StateStore,
|
||||||
) -> None:
|
) -> None:
|
||||||
store.append_event(AssistantText(text="secret repo context", model="m"))
|
log(store).append(AssistantText(text="secret repo context", model="m"))
|
||||||
client = ScriptedClient([GOOD])
|
client = ScriptedClient([GOOD])
|
||||||
desk(client, store).ask("Is this API deprecated?")
|
desk(client, store).ask("Is this API deprecated?")
|
||||||
|
|
||||||
@@ -134,7 +151,7 @@ def test_consults_are_recorded_as_composite_events(store: StateStore) -> None:
|
|||||||
# L9 wants a consult/delegation eval set; reusing the pair gets consult
|
# L9 wants a consult/delegation eval set; reusing the pair gets consult
|
||||||
# evaluation for free.
|
# evaluation for free.
|
||||||
desk(ScriptedClient([GOOD]), store).ask("q")
|
desk(ScriptedClient([GOOD]), store).ask("q")
|
||||||
events = [e.event for e in store.read_events()]
|
events = [e.event for e in log(store).read()]
|
||||||
started = next(e for e in events if isinstance(e, DelegationStarted))
|
started = next(e for e in events if isinstance(e, DelegationStarted))
|
||||||
completed = next(e for e in events if isinstance(e, DelegationCompleted))
|
completed = next(e for e in events if isinstance(e, DelegationCompleted))
|
||||||
assert started.role == "expert"
|
assert started.role == "expert"
|
||||||
@@ -146,7 +163,7 @@ def test_a_failed_consult_is_recorded_too(store: StateStore) -> None:
|
|||||||
with pytest.raises(LLMError):
|
with pytest.raises(LLMError):
|
||||||
desk(ScriptedClient(["waffle", "waffle"]), store).ask("q")
|
desk(ScriptedClient(["waffle", "waffle"]), store).ask("q")
|
||||||
completed = [
|
completed = [
|
||||||
e.event for e in store.read_events() if isinstance(e.event, DelegationCompleted)
|
e.event for e in log(store).read() if isinstance(e.event, DelegationCompleted)
|
||||||
]
|
]
|
||||||
assert completed[0].exit_status == "failed"
|
assert completed[0].exit_status == "failed"
|
||||||
|
|
||||||
@@ -177,10 +194,18 @@ def test_budget_survives_a_restart(tmp_path: Path) -> None:
|
|||||||
def test_budget_counts_only_after_the_task_marker(store: StateStore) -> None:
|
def test_budget_counts_only_after_the_task_marker(store: StateStore) -> None:
|
||||||
d = desk(ScriptedClient([GOOD] * 5), store, budget=1)
|
d = desk(ScriptedClient([GOOD] * 5), store, budget=1)
|
||||||
d.ask("earlier task")
|
d.ask("earlier task")
|
||||||
marker = store.read_events()[-1].id
|
marker = log(store).read()[-1].id
|
||||||
# A new task starts its allowance afresh from its own marker.
|
# A new task starts its allowance afresh from its own marker.
|
||||||
assert d.ask("new task", since_event_id=marker).confidence == "high"
|
assert d.ask("new task", since_event_id=marker).confidence == "high"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_budget_is_per_work_item(store: StateStore) -> None:
|
||||||
|
# Consults in one task never use up another's allowance (L9).
|
||||||
|
desk(ScriptedClient([GOOD]), store, item="a", budget=1).ask("one")
|
||||||
|
assert desk(ScriptedClient([GOOD]), store, item="b", budget=1).ask("two")
|
||||||
|
with pytest.raises(BudgetExhausted):
|
||||||
|
desk(ScriptedClient([GOOD]), store, item="a", budget=1).ask("three")
|
||||||
|
|
||||||
|
|
||||||
def test_default_budget_is_small() -> None:
|
def test_default_budget_is_small() -> None:
|
||||||
assert DEFAULT_BUDGET <= 5 # L9: do not let this become a second subsystem
|
assert DEFAULT_BUDGET <= 5 # L9: do not let this become a second subsystem
|
||||||
@@ -62,6 +62,7 @@ class Planner:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: object,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -74,7 +75,7 @@ class Executor:
|
|||||||
def __init__(self, faults: Faults) -> None:
|
def __init__(self, faults: Faults) -> None:
|
||||||
self._faults = faults
|
self._faults = faults
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
return self._faults.check("executor") or ExecutorOutcome("completed", "done")
|
return self._faults.check("executor") or ExecutorOutcome("completed", "done")
|
||||||
|
|
||||||
|
|
||||||
@@ -189,7 +190,7 @@ def test_a_signoff_state_without_its_gate_offers_a_way_out(tmp_path: Path) -> No
|
|||||||
|
|
||||||
|
|
||||||
class RaisingReviewExecutor:
|
class RaisingReviewExecutor:
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
raise RuntimeError("bug while addressing review")
|
raise RuntimeError("bug while addressing review")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -45,9 +45,9 @@ def test_transition_logs_from_to_and_cause(
|
|||||||
) -> None:
|
) -> None:
|
||||||
store.transition("PLANNING_ARCH", cause="operator command: use demo", project="d")
|
store.transition("PLANNING_ARCH", cause="operator command: use demo", project="d")
|
||||||
store.transition("AWAIT_ARCH_SIGNOFF", cause="plan submitted", project="d")
|
store.transition("AWAIT_ARCH_SIGNOFF", cause="plan submitted", project="d")
|
||||||
first, second = events(stream)
|
# Opening the work item logs its own events first (work_item.open, .switch).
|
||||||
assert first["event"] == "state.transition"
|
first, second = [e for e in events(stream) if e["event"] == "state.transition"]
|
||||||
assert (first["from"], first["to"]) == (None, "PLANNING_ARCH")
|
assert (first["from"], first["to"]) == ("IDLE", "PLANNING_ARCH")
|
||||||
assert (second["from"], second["to"]) == ("PLANNING_ARCH", "AWAIT_ARCH_SIGNOFF")
|
assert (second["from"], second["to"]) == ("PLANNING_ARCH", "AWAIT_ARCH_SIGNOFF")
|
||||||
assert second["cause"] == "plan submitted"
|
assert second["cause"] == "plan submitted"
|
||||||
|
|
||||||
@@ -66,9 +66,9 @@ def test_task_id_threads_through_events(store: StateStore, stream: io.StringIO)
|
|||||||
store.transition("IDLE", cause="boot")
|
store.transition("IDLE", cause="boot")
|
||||||
store.oplog.task_id = "task-42"
|
store.oplog.task_id = "task-42"
|
||||||
store.transition("PLANNING_ARCH", cause="operator command")
|
store.transition("PLANNING_ARCH", cause="operator command")
|
||||||
before, during = events(stream)
|
before, *during = events(stream)
|
||||||
assert before["task_id"] is None
|
assert before["task_id"] is None
|
||||||
assert during["task_id"] == "task-42"
|
assert {e["task_id"] for e in during} == {"task-42"}
|
||||||
|
|
||||||
|
|
||||||
def test_migration_emits_event(tmp_path: Path, stream: io.StringIO) -> None:
|
def test_migration_emits_event(tmp_path: Path, stream: io.StringIO) -> None:
|
||||||
@@ -120,7 +120,10 @@ def test_literal_log_lines_for_human_readability(
|
|||||||
monkeypatch.setattr("nisse_dev.oplog.datetime", _FrozenDatetime)
|
monkeypatch.setattr("nisse_dev.oplog.datetime", _FrozenDatetime)
|
||||||
|
|
||||||
store.transition(
|
store.transition(
|
||||||
"PLANNING_ARCH", cause="operator command: use demo", project="demo"
|
"PLANNING_ARCH",
|
||||||
|
cause="operator command: use demo",
|
||||||
|
project="demo",
|
||||||
|
data={"task_id": "demo-m2"},
|
||||||
)
|
)
|
||||||
store.oplog.task_id = "demo-m2"
|
store.oplog.task_id = "demo-m2"
|
||||||
intent_id = store.begin_intent(
|
intent_id = store.begin_intent(
|
||||||
@@ -129,11 +132,23 @@ def test_literal_log_lines_for_human_readability(
|
|||||||
store.commit_intent(intent_id, cause="push verified by Gitea query")
|
store.commit_intent(intent_id, cause="push verified by Gitea query")
|
||||||
|
|
||||||
assert stream.getvalue().splitlines() == [
|
assert stream.getvalue().splitlines() == [
|
||||||
|
(
|
||||||
|
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
||||||
|
' "event": "work_item.open", "task_id": null,'
|
||||||
|
' "cause": "operator command: use demo",'
|
||||||
|
' "work_item": "demo-m2", "scope": "demo"}'
|
||||||
|
),
|
||||||
|
(
|
||||||
|
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
||||||
|
' "event": "work_item.switch", "task_id": null,'
|
||||||
|
' "cause": "operator command: use demo",'
|
||||||
|
' "from": null, "to": "demo-m2"}'
|
||||||
|
),
|
||||||
(
|
(
|
||||||
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
||||||
' "event": "state.transition", "task_id": null,'
|
' "event": "state.transition", "task_id": null,'
|
||||||
' "cause": "operator command: use demo",'
|
' "cause": "operator command: use demo",'
|
||||||
' "from": null, "to": "PLANNING_ARCH", "project": "demo"}'
|
' "from": "IDLE", "to": "PLANNING_ARCH", "project": "demo"}'
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
'{"ts": "2026-09-05T12:00:00+00:00", "agent": "nisse-dev",'
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ from nisse_dev.control import (
|
|||||||
from nisse_dev.llm import BudgetExhausted, RoleMap
|
from nisse_dev.llm import BudgetExhausted, RoleMap
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.planner import PLANNER_TOOLS, LLMPlanningAgent
|
from nisse_dev.planner import PLANNER_TOOLS, LLMPlanningAgent
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
from nisse_dev.transport import InMemoryTransport
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
ROLES = RoleMap(
|
ROLES = RoleMap(
|
||||||
@@ -64,6 +64,13 @@ def plan_call(content: str, gate: str = "arch") -> ToolCall:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
ITEM = "t" # the work item the unit tests plan in
|
||||||
|
|
||||||
|
|
||||||
|
def log(store: StateStore) -> Transcript:
|
||||||
|
return store.transcript(ITEM, "planner")
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def store(tmp_path: Path) -> StateStore:
|
def store(tmp_path: Path) -> StateStore:
|
||||||
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
@@ -71,8 +78,8 @@ def store(tmp_path: Path) -> StateStore:
|
|||||||
|
|
||||||
def test_submit_plan_tool_call_becomes_submit_plan(store: StateStore) -> None:
|
def test_submit_plan_tool_call_becomes_submit_plan(store: StateStore) -> None:
|
||||||
client = ScriptedClient([[plan_call("step 1: create src/app.py")]])
|
client = ScriptedClient([[plan_call("step 1: create src/app.py")]])
|
||||||
agent = LLMPlanningAgent(client, store, ROLES)
|
agent = LLMPlanningAgent(client, ROLES)
|
||||||
action = agent.plan("arch", "demo")
|
action = agent.plan(log(store), "arch", "demo")
|
||||||
assert action == SubmitPlan("step 1: create src/app.py")
|
assert action == SubmitPlan("step 1: create src/app.py")
|
||||||
|
|
||||||
|
|
||||||
@@ -88,13 +95,13 @@ def test_ask_human_tool_call_becomes_ask_human(store: StateStore) -> None:
|
|||||||
]
|
]
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
action = LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
action = LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
assert action == AskHuman("postgres or sqlite?")
|
assert action == AskHuman("postgres or sqlite?")
|
||||||
|
|
||||||
|
|
||||||
def test_planner_role_and_tools_are_used(store: StateStore) -> None:
|
def test_planner_role_and_tools_are_used(store: StateStore) -> None:
|
||||||
client = ScriptedClient([[plan_call("plan")]])
|
client = ScriptedClient([[plan_call("plan")]])
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
call = client.calls[0]
|
call = client.calls[0]
|
||||||
assert call["model"] == "anthropic/claude-opus-5" # never the executor
|
assert call["model"] == "anthropic/claude-opus-5" # never the executor
|
||||||
assert [t["function"]["name"] for t in call["tools"]] == [
|
assert [t["function"]["name"] for t in call["tools"]] == [
|
||||||
@@ -113,8 +120,8 @@ def test_model_output_is_appended_to_the_log(store: StateStore) -> None:
|
|||||||
client = ScriptedClient(
|
client = ScriptedClient(
|
||||||
[[AssistantText(text="thinking out loud", model="m"), plan_call("the plan")]]
|
[[AssistantText(text="thinking out loud", model="m"), plan_call("the plan")]]
|
||||||
)
|
)
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
kinds = [type(e.event).__name__ for e in store.read_events()]
|
kinds = [type(e.event).__name__ for e in log(store).read()]
|
||||||
assert kinds == ["AssistantText", "ToolCall", "ToolResult"]
|
assert kinds == ["AssistantText", "ToolCall", "ToolResult"]
|
||||||
|
|
||||||
|
|
||||||
@@ -122,8 +129,8 @@ def test_gate_tool_call_is_answered_in_the_log(store: StateStore) -> None:
|
|||||||
# submit_plan ends the turn and has no natural result, but a tool call
|
# submit_plan ends the turn and has no natural result, but a tool call
|
||||||
# left dangling makes the next render invalid for the provider.
|
# left dangling makes the next render invalid for the provider.
|
||||||
client = ScriptedClient([[plan_call("the plan")]])
|
client = ScriptedClient([[plan_call("the plan")]])
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
events = [e.event for e in store.read_events()]
|
events = [e.event for e in log(store).read()]
|
||||||
assert [type(e).__name__ for e in events] == ["ToolCall", "ToolResult"]
|
assert [type(e).__name__ for e in events] == ["ToolCall", "ToolResult"]
|
||||||
assert events[1].call_id == events[0].call_id
|
assert events[1].call_id == events[0].call_id
|
||||||
assert "posted for approval" in events[1].result
|
assert "posted for approval" in events[1].result
|
||||||
@@ -137,7 +144,7 @@ def test_prose_only_turn_retries_once_then_raises(store: StateStore) -> None:
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
with pytest.raises(PlanningError, match="no submit_plan or ask_human"):
|
with pytest.raises(PlanningError, match="no submit_plan or ask_human"):
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
assert len(client.calls) == 2
|
assert len(client.calls) == 2
|
||||||
assert "did not call a tool" in client.calls[1]["system"]
|
assert "did not call a tool" in client.calls[1]["system"]
|
||||||
|
|
||||||
@@ -146,7 +153,7 @@ def test_retry_succeeds_after_prose(store: StateStore) -> None:
|
|||||||
client = ScriptedClient(
|
client = ScriptedClient(
|
||||||
[[AssistantText(text="prose", model="m")], [plan_call("the real plan")]]
|
[[AssistantText(text="prose", model="m")], [plan_call("the real plan")]]
|
||||||
)
|
)
|
||||||
action = LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
action = LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
assert action == SubmitPlan("the real plan")
|
assert action == SubmitPlan("the real plan")
|
||||||
|
|
||||||
|
|
||||||
@@ -154,13 +161,15 @@ def test_malformed_tool_arguments_are_rejected(store: StateStore) -> None:
|
|||||||
blank = ToolCall(call_id="c", tool="submit_plan", arguments={"content": " "})
|
blank = ToolCall(call_id="c", tool="submit_plan", arguments={"content": " "})
|
||||||
client = ScriptedClient([[blank], [blank]])
|
client = ScriptedClient([[blank], [blank]])
|
||||||
with pytest.raises(PlanningError):
|
with pytest.raises(PlanningError):
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
|
|
||||||
|
|
||||||
def test_planner_sees_prior_transcript(store: StateStore) -> None:
|
def test_planner_sees_prior_transcript(store: StateStore) -> None:
|
||||||
store.append_event(UserMessage(text="use demo", source="transport:@don"))
|
store.transcript(ITEM, "operator").append(
|
||||||
|
UserMessage(text="use demo", source="transport:@don")
|
||||||
|
)
|
||||||
client = ScriptedClient([[plan_call("plan")]])
|
client = ScriptedClient([[plan_call("plan")]])
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
seen = [e.event for e in client.calls[0]["events"]]
|
seen = [e.event for e in client.calls[0]["events"]]
|
||||||
assert UserMessage(text="use demo", source="transport:@don") in seen
|
assert UserMessage(text="use demo", source="transport:@don") in seen
|
||||||
|
|
||||||
@@ -171,7 +180,7 @@ def test_planner_sees_prior_transcript(store: StateStore) -> None:
|
|||||||
def build(tmp_path: Path, client: ScriptedClient) -> tuple[Orchestrator, StateStore]:
|
def build(tmp_path: Path, client: ScriptedClient) -> tuple[Orchestrator, StateStore]:
|
||||||
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
transport = InMemoryTransport()
|
transport = InMemoryTransport()
|
||||||
agent = LLMPlanningAgent(client, store, ROLES)
|
agent = LLMPlanningAgent(client, ROLES)
|
||||||
orch = Orchestrator(store, transport, agent, operators={"don"}, roles=ROLES)
|
orch = Orchestrator(store, transport, agent, operators={"don"}, roles=ROLES)
|
||||||
orch.start()
|
orch.start()
|
||||||
return orch, store, transport
|
return orch, store, transport
|
||||||
@@ -186,7 +195,10 @@ def test_use_project_produces_llm_plan_and_transcript(tmp_path: Path) -> None:
|
|||||||
assert request.kind == "approval_request"
|
assert request.kind == "approval_request"
|
||||||
assert "create app.py" in request.text
|
assert "create app.py" in request.text
|
||||||
|
|
||||||
events = [e.event for e in store.read_events()]
|
# The work item's own events: `use demo` was said while idle, and is
|
||||||
|
# carried into the item it opened (D1).
|
||||||
|
item = store.active_item()
|
||||||
|
events = [e.event for e in store.transcript(item.id, "operator").read()]
|
||||||
kinds = [type(e).__name__ for e in events]
|
kinds = [type(e).__name__ for e in events]
|
||||||
assert kinds == [
|
assert kinds == [
|
||||||
"UserMessage",
|
"UserMessage",
|
||||||
@@ -237,7 +249,7 @@ def test_operator_can_recover_from_a_planning_failure(tmp_path: Path) -> None:
|
|||||||
|
|
||||||
def test_operator_can_recover_after_a_stopped_milestone(tmp_path: Path) -> None:
|
def test_operator_can_recover_after_a_stopped_milestone(tmp_path: Path) -> None:
|
||||||
class StoppedExecutor:
|
class StoppedExecutor:
|
||||||
def run(self, milestone: int) -> Any:
|
def run(self, transcript: object, milestone: int) -> Any:
|
||||||
from nisse_dev.executor import ExecutorOutcome
|
from nisse_dev.executor import ExecutorOutcome
|
||||||
|
|
||||||
return ExecutorOutcome("escalate", "tests failed twice")
|
return ExecutorOutcome("escalate", "tests failed twice")
|
||||||
@@ -248,7 +260,7 @@ def test_operator_can_recover_after_a_stopped_milestone(tmp_path: Path) -> None:
|
|||||||
orch = Orchestrator(
|
orch = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(client, store, ROLES),
|
LLMPlanningAgent(client, ROLES),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=ROLES,
|
roles=ROLES,
|
||||||
executor=StoppedExecutor(),
|
executor=StoppedExecutor(),
|
||||||
@@ -295,5 +307,5 @@ def test_budget_exhaustion_is_not_retried(store: StateStore) -> None:
|
|||||||
|
|
||||||
client = BrokeClient()
|
client = BrokeClient()
|
||||||
with pytest.raises(BudgetExhausted):
|
with pytest.raises(BudgetExhausted):
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, ROLES).plan(log(store), "arch", "demo")
|
||||||
assert client.calls == 1 # not two
|
assert client.calls == 1 # not two
|
||||||
@@ -57,7 +57,7 @@ class ScriptedClient:
|
|||||||
|
|
||||||
|
|
||||||
class DoneExecutor:
|
class DoneExecutor:
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
return ExecutorOutcome("completed", f"milestone {milestone} done")
|
return ExecutorOutcome("completed", f"milestone {milestone} done")
|
||||||
|
|
||||||
|
|
||||||
@@ -91,7 +91,7 @@ def build(
|
|||||||
orch = Orchestrator(
|
orch = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(client, store, ROLES),
|
LLMPlanningAgent(client, ROLES),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=ROLES,
|
roles=ROLES,
|
||||||
executor=DoneExecutor(),
|
executor=DoneExecutor(),
|
||||||
@@ -183,7 +183,7 @@ def test_publisher_reconciler_is_registered_at_boot(tmp_path: Path) -> None:
|
|||||||
orch = Orchestrator(
|
orch = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(ScriptedClient([]), store, ROLES),
|
LLMPlanningAgent(ScriptedClient([]), ROLES),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=ROLES,
|
roles=ROLES,
|
||||||
publisher=FakePublisher(),
|
publisher=FakePublisher(),
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ class Executor:
|
|||||||
self.runs = 0
|
self.runs = 0
|
||||||
self.outcome = ExecutorOutcome("completed", "done")
|
self.outcome = ExecutorOutcome("completed", "done")
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
self.runs += 1
|
self.runs += 1
|
||||||
return self.outcome
|
return self.outcome
|
||||||
|
|
||||||
|
|||||||
@@ -104,7 +104,7 @@ class Executor:
|
|||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.outcome = ExecutorOutcome("completed", "done")
|
self.outcome = ExecutorOutcome("completed", "done")
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
return self.outcome
|
return self.outcome
|
||||||
|
|
||||||
|
|
||||||
@@ -283,7 +283,7 @@ def test_crash_during_a_review_loses_the_comment(tmp_path: Path) -> None:
|
|||||||
class Crash(BaseException):
|
class Crash(BaseException):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def dies(milestone: int) -> ExecutorOutcome:
|
def dies(transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
raise Crash() # the process dies mid-review
|
raise Crash() # the process dies mid-review
|
||||||
|
|
||||||
agent.executor.run = dies # type: ignore[method-assign]
|
agent.executor.run = dies # type: ignore[method-assign]
|
||||||
@@ -348,7 +348,7 @@ class Crash(BaseException):
|
|||||||
|
|
||||||
|
|
||||||
def crash_mid_review(agent: Agent, gitea: FakeGitea) -> None:
|
def crash_mid_review(agent: Agent, gitea: FakeGitea) -> None:
|
||||||
def dies(milestone: int) -> ExecutorOutcome:
|
def dies(transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
raise Crash()
|
raise Crash()
|
||||||
|
|
||||||
agent.executor.run = dies # type: ignore[method-assign]
|
agent.executor.run = dies # type: ignore[method-assign]
|
||||||
@@ -377,7 +377,7 @@ def test_an_executor_error_answers_the_comment_so_it_is_not_redelivered(
|
|||||||
agent.say("use demo", "approve", "approve", "start milestone 1")
|
agent.say("use demo", "approve", "approve", "start milestone 1")
|
||||||
gitea.add(20, "don", "please add a test")
|
gitea.add(20, "don", "please add a test")
|
||||||
|
|
||||||
def fails(milestone: int) -> ExecutorOutcome:
|
def fails(transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
raise RuntimeError("bug in the executor")
|
raise RuntimeError("bug in the executor")
|
||||||
|
|
||||||
agent.executor.run = fails # type: ignore[method-assign]
|
agent.executor.run = fails # type: ignore[method-assign]
|
||||||
|
|||||||
@@ -42,10 +42,10 @@ class RecordingExecutor:
|
|||||||
self._outcomes = outcomes or []
|
self._outcomes = outcomes or []
|
||||||
self.store: StateStore | None = None
|
self.store: StateStore | None = None
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: Any, milestone: int) -> ExecutorOutcome:
|
||||||
self.runs.append(milestone)
|
self.runs.append(milestone)
|
||||||
if self.store is not None:
|
# What it is given: the work item's transcript, nothing wider.
|
||||||
self.seen_events.append(self.store.read_events())
|
self.seen_events.append(transcript.read())
|
||||||
status = self._outcomes.pop(0) if self._outcomes else "completed"
|
status = self._outcomes.pop(0) if self._outcomes else "completed"
|
||||||
return ExecutorOutcome(status, f"executor said {status}")
|
return ExecutorOutcome(status, f"executor said {status}")
|
||||||
|
|
||||||
@@ -88,7 +88,7 @@ def build(
|
|||||||
orch = Orchestrator(
|
orch = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(ScriptedClient(), store, ROLES),
|
LLMPlanningAgent(ScriptedClient(), ROLES),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=ROLES,
|
roles=ROLES,
|
||||||
executor=executor,
|
executor=executor,
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ from nisse_dev.executor import ExecutorOutcome, LLMExecutor
|
|||||||
from nisse_dev.llm import RoleMap
|
from nisse_dev.llm import RoleMap
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.planner import LLMPlanningAgent
|
from nisse_dev.planner import LLMPlanningAgent
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore, Transcript
|
||||||
from nisse_dev.tools import ToolOutcome
|
from nisse_dev.tools import ToolOutcome
|
||||||
from nisse_dev.transport import InMemoryTransport
|
from nisse_dev.transport import InMemoryTransport
|
||||||
from nisse_dev.worker import SerialWorker
|
from nisse_dev.worker import SerialWorker
|
||||||
@@ -36,13 +36,17 @@ ROLES = RoleMap(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def log(store: StateStore, actor: str = "executor") -> Transcript:
|
||||||
|
return store.transcript("t", actor)
|
||||||
|
|
||||||
|
|
||||||
def new_store(path: Path) -> StateStore:
|
def new_store(path: Path) -> StateStore:
|
||||||
return StateStore(path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
return StateStore(path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
def user_messages(store: StateStore) -> list[str]:
|
def user_messages(store: StateStore) -> list[str]:
|
||||||
return [
|
return [
|
||||||
e.event.text for e in store.read_events() if isinstance(e.event, UserMessage)
|
e.event.text for e in store.all_events() if isinstance(e.event, UserMessage)
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
@@ -58,7 +62,7 @@ class OperatorDuringWork:
|
|||||||
self.says = says
|
self.says = says
|
||||||
self.transport: InMemoryTransport | None = None
|
self.transport: InMemoryTransport | None = None
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
assert self.transport is not None
|
assert self.transport is not None
|
||||||
self.transport.receive("don", self.says)
|
self.transport.receive("don", self.says)
|
||||||
if self.cancel.requested:
|
if self.cancel.requested:
|
||||||
@@ -158,6 +162,7 @@ class StopsBeforePlanning:
|
|||||||
|
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
|
transcript: object,
|
||||||
gate: str,
|
gate: str,
|
||||||
project: str,
|
project: str,
|
||||||
feedback: str | None = None,
|
feedback: str | None = None,
|
||||||
@@ -200,7 +205,7 @@ class BlocksUntilStopped:
|
|||||||
self.cancel = cancel
|
self.cancel = cancel
|
||||||
self.running = threading.Event()
|
self.running = threading.Event()
|
||||||
|
|
||||||
def run(self, milestone: int) -> ExecutorOutcome:
|
def run(self, transcript: object, milestone: int) -> ExecutorOutcome:
|
||||||
self.running.set()
|
self.running.set()
|
||||||
for _ in range(500):
|
for _ in range(500):
|
||||||
if self.cancel.requested:
|
if self.cancel.requested:
|
||||||
@@ -287,10 +292,8 @@ def test_executor_stops_before_a_turn_without_calling_the_model(
|
|||||||
cancel = Cancel()
|
cancel = Cancel()
|
||||||
cancel.set()
|
cancel.set()
|
||||||
client = ScriptedClient([])
|
client = ScriptedClient([])
|
||||||
executor = LLMExecutor(
|
executor = LLMExecutor(client, ROLES, StopsAfterFirstCall(cancel), cancel=cancel)
|
||||||
client, new_store(tmp_path), ROLES, StopsAfterFirstCall(cancel), cancel=cancel
|
outcome = executor.run(log(new_store(tmp_path)), 1)
|
||||||
)
|
|
||||||
outcome = executor.run(1)
|
|
||||||
assert outcome == ExecutorOutcome("stopped", "before turn 1")
|
assert outcome == ExecutorOutcome("stopped", "before turn 1")
|
||||||
assert client.calls == 0
|
assert client.calls == 0
|
||||||
|
|
||||||
@@ -305,14 +308,14 @@ def test_executor_stops_between_tool_calls_and_records_the_skipped_ones(
|
|||||||
ToolCall(call_id="a", tool="run_command", arguments={"command": "make"}),
|
ToolCall(call_id="a", tool="run_command", arguments={"command": "make"}),
|
||||||
ToolCall(call_id="b", tool="write_file", arguments={"path": "x"}),
|
ToolCall(call_id="b", tool="write_file", arguments={"path": "x"}),
|
||||||
]
|
]
|
||||||
executor = LLMExecutor(ScriptedClient([calls]), store, ROLES, ws, cancel=cancel)
|
executor = LLMExecutor(ScriptedClient([calls]), ROLES, ws, cancel=cancel)
|
||||||
outcome = executor.run(1)
|
outcome = executor.run(log(store), 1)
|
||||||
|
|
||||||
assert outcome == ExecutorOutcome("stopped", "during turn 1")
|
assert outcome == ExecutorOutcome("stopped", "during turn 1")
|
||||||
assert ws.ran == ["run_command"]
|
assert ws.ran == ["run_command"]
|
||||||
results = {
|
results = {
|
||||||
e.event.call_id: e.event.result
|
e.event.call_id: e.event.result
|
||||||
for e in store.read_events()
|
for e in log(store).read()
|
||||||
if isinstance(e.event, ToolResult)
|
if isinstance(e.event, ToolResult)
|
||||||
}
|
}
|
||||||
# Every call has a result, so the transcript still renders.
|
# Every call has a result, so the transcript still renders.
|
||||||
@@ -320,7 +323,7 @@ def test_executor_stops_between_tool_calls_and_records_the_skipped_ones(
|
|||||||
|
|
||||||
|
|
||||||
class StoppedDelegate:
|
class StoppedDelegate:
|
||||||
def run(self, *, brief: str, branch: str) -> DelegationResult:
|
def run(self, transcript: object, *, brief: str, branch: str) -> DelegationResult:
|
||||||
return DelegationResult("stopped", "stopped by the operator during turn 2")
|
return DelegationResult("stopped", "stopped by the operator during turn 2")
|
||||||
|
|
||||||
|
|
||||||
@@ -339,12 +342,11 @@ def test_a_stopped_delegation_stops_the_milestone(tmp_path: Path) -> None:
|
|||||||
|
|
||||||
executor = LLMExecutor(
|
executor = LLMExecutor(
|
||||||
ScriptedClient(failing),
|
ScriptedClient(failing),
|
||||||
new_store(tmp_path),
|
|
||||||
ROLES,
|
ROLES,
|
||||||
Failing(),
|
Failing(),
|
||||||
delegate=StoppedDelegate(),
|
delegate=StoppedDelegate(),
|
||||||
)
|
)
|
||||||
outcome = executor.run(1)
|
outcome = executor.run(log(new_store(tmp_path)), 1)
|
||||||
assert outcome.status == "stopped"
|
assert outcome.status == "stopped"
|
||||||
assert "during a delegation" in outcome.note
|
assert "during a delegation" in outcome.note
|
||||||
|
|
||||||
@@ -353,9 +355,9 @@ def test_planner_does_not_start_a_turn_once_stopped(tmp_path: Path) -> None:
|
|||||||
cancel = Cancel()
|
cancel = Cancel()
|
||||||
cancel.set()
|
cancel.set()
|
||||||
client = ScriptedClient([])
|
client = ScriptedClient([])
|
||||||
planner = LLMPlanningAgent(client, new_store(tmp_path), ROLES, cancel=cancel)
|
planner = LLMPlanningAgent(client, ROLES, cancel=cancel)
|
||||||
try:
|
try:
|
||||||
planner.plan("arch", "demo")
|
planner.plan(log(new_store(tmp_path), "planner"), "arch", "demo")
|
||||||
except PlanningStopped:
|
except PlanningStopped:
|
||||||
pass
|
pass
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -0,0 +1,177 @@
|
|||||||
|
"""Work items in the state store (architecture L1, L7; Development Plan M1.1)."""
|
||||||
|
|
||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import AssistantText, UserMessage
|
||||||
|
from nisse_dev.control import (
|
||||||
|
AWAIT_ARCH_SIGNOFF,
|
||||||
|
PR_OPEN,
|
||||||
|
AskHuman,
|
||||||
|
Orchestrator,
|
||||||
|
SubmitPlan,
|
||||||
|
)
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.state import IDLE, StateStore
|
||||||
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
|
|
||||||
|
def new_store(path: Path) -> StateStore:
|
||||||
|
return StateStore(path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
|
# --- the store ---------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_transcript_reads_only_its_own_work_item(tmp_path: Path) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
store.transcript("a", "operator").append(UserMessage(text="for a", source="m"))
|
||||||
|
store.transcript("b", "operator").append(UserMessage(text="for b", source="m"))
|
||||||
|
store.transcript(None, "operator").append(UserMessage(text="idle", source="m"))
|
||||||
|
assert [e.event.text for e in store.transcript("a", "planner").read()] == ["for a"]
|
||||||
|
assert [e.event.text for e in store.transcript(None, "planner").read()] == ["idle"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_every_event_carries_its_work_item_and_actor(tmp_path: Path) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
log = store.transcript("a", "executor")
|
||||||
|
log.append(AssistantText(text="working", model="m"))
|
||||||
|
log.as_actor("helper").append(AssistantText(text="helping", model="f"))
|
||||||
|
assert [(e.work_item, e.actor) for e in store.all_events()] == [
|
||||||
|
("a", "executor"),
|
||||||
|
("a", "helper"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_actor_is_refused(tmp_path: Path) -> None:
|
||||||
|
with pytest.raises(ValueError, match="unknown actor"):
|
||||||
|
new_store(tmp_path).transcript("a", "someone")
|
||||||
|
|
||||||
|
|
||||||
|
def test_intents_are_owned_by_their_work_item(tmp_path: Path) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
store.transcript("a", "helper").begin_intent("delegation", {}, cause="t")
|
||||||
|
[intent] = store.pending_intents()
|
||||||
|
assert intent.work_item == "a"
|
||||||
|
|
||||||
|
|
||||||
|
def test_work_items_and_the_active_one_survive_a_restart(tmp_path: Path) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
store.open_work_item("a", title="demo", scope="demo", position=IDLE, cause="t")
|
||||||
|
store.set_active("a", cause="t")
|
||||||
|
store.update_work_item("a", position="PLANNING_ARCH", data={"x": 1}, cause="t")
|
||||||
|
store.close()
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
item = store.active_item()
|
||||||
|
assert item is not None
|
||||||
|
assert (item.id, item.position, item.data, item.status) == (
|
||||||
|
"a",
|
||||||
|
"PLANNING_ARCH",
|
||||||
|
{"x": 1},
|
||||||
|
"open",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_closing_records_the_outcome_and_going_idle_clears_active(
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
store.open_work_item("a", title="demo", position=IDLE, cause="t")
|
||||||
|
store.set_active("a", cause="t")
|
||||||
|
store.close_work_item("a", outcome="completed", cause="t")
|
||||||
|
store.set_active(None, cause="t")
|
||||||
|
assert store.active_item() is None
|
||||||
|
item = store.work_item("a")
|
||||||
|
assert item is not None and (item.status, item.outcome) == ("closed", "completed")
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
store.close_work_item("a", outcome="abandoned", cause="t") # already closed
|
||||||
|
|
||||||
|
|
||||||
|
def test_messages_sent_while_idle_are_carried_with_their_origin(
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
store.set_active(None, cause="boot")
|
||||||
|
idle = store.transcript(None, "operator")
|
||||||
|
first = idle.append(UserMessage(text="add a truncate helper", source="m:1"))
|
||||||
|
store.open_work_item("a", title="demo", position=IDLE, cause="t")
|
||||||
|
assert store.carry_idle_messages("a") == 1
|
||||||
|
[carried] = store.transcript("a", "planner").read()
|
||||||
|
assert carried.event.text == "add a truncate helper"
|
||||||
|
assert f"#{first}" in carried.event.source # where it came from
|
||||||
|
assert len(store.all_events()) == 2 # a copy; the original is untouched
|
||||||
|
|
||||||
|
|
||||||
|
# --- the orchestrator ----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingPlanner:
|
||||||
|
"""Records what each planning turn could read."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.seen: list[list[str]] = []
|
||||||
|
|
||||||
|
def plan(
|
||||||
|
self,
|
||||||
|
transcript: Any,
|
||||||
|
gate: str,
|
||||||
|
project: str,
|
||||||
|
feedback: str | None = None,
|
||||||
|
answer: str | None = None,
|
||||||
|
) -> SubmitPlan | AskHuman:
|
||||||
|
self.seen.append(
|
||||||
|
[
|
||||||
|
e.event.text
|
||||||
|
for e in transcript.read()
|
||||||
|
if isinstance(e.event, UserMessage)
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return SubmitPlan(f"{gate} plan for {project}")
|
||||||
|
|
||||||
|
|
||||||
|
def build(tmp_path: Path) -> tuple[StateStore, InMemoryTransport, RecordingPlanner]:
|
||||||
|
store = new_store(tmp_path)
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
planner = RecordingPlanner()
|
||||||
|
Orchestrator(store, transport, planner, operators={"don"}).start()
|
||||||
|
return store, transport, planner
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_second_task_never_sees_the_first(tmp_path: Path) -> None:
|
||||||
|
# The second live run's planner read every earlier task's events (#78).
|
||||||
|
store, transport, planner = build(tmp_path)
|
||||||
|
for text in ("build the first thing", "use first", "approve", "approve"):
|
||||||
|
transport.receive("don", text)
|
||||||
|
transport.receive("don", "start milestone 1")
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
|
transport.receive("don", "project complete")
|
||||||
|
|
||||||
|
transport.receive("don", "build the second thing")
|
||||||
|
transport.receive("don", "use second")
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
second_task_view = planner.seen[-1]
|
||||||
|
assert "build the second thing" in second_task_view
|
||||||
|
assert not any("first" in text for text in second_task_view)
|
||||||
|
|
||||||
|
|
||||||
|
def test_each_task_is_its_own_work_item_with_its_outcome(tmp_path: Path) -> None:
|
||||||
|
store, transport, _ = build(tmp_path)
|
||||||
|
for text in ("use first", "approve", "approve", "start milestone 1"):
|
||||||
|
transport.receive("don", text)
|
||||||
|
transport.receive("don", "project complete")
|
||||||
|
transport.receive("don", "use second")
|
||||||
|
transport.receive("don", "abandon")
|
||||||
|
outcomes = [(item.title, item.outcome) for item in store.work_items()]
|
||||||
|
assert outcomes == [("first", "completed"), ("second", "abandoned")]
|
||||||
|
assert store.active_item() is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_task_message_before_use_reaches_the_planner(tmp_path: Path) -> None:
|
||||||
|
# Run 2's flow: the task, then `use`. It must still be planned from.
|
||||||
|
_, transport, planner = build(tmp_path)
|
||||||
|
transport.receive("don", "add a truncate helper")
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
assert "add a truncate helper" in planner.seen[0]
|
||||||
@@ -21,7 +21,7 @@ from nisse_dev.state import StateStore
|
|||||||
store = StateStore(sys.argv[1])
|
store = StateStore(sys.argv[1])
|
||||||
store.transition("EXECUTING", cause="test", project="demo", data={"milestone": 2})
|
store.transition("EXECUTING", cause="test", project="demo", data={"milestone": 2})
|
||||||
store.begin_intent("push_branch", {"branch": "milestone-2"}, cause="test")
|
store.begin_intent("push_branch", {"branch": "milestone-2"}, cause="test")
|
||||||
store.append_event(UserMessage(text="start milestone 2", source="m:1"))
|
store.transcript("t", "operator").append(UserMessage(text="start milestone 2", source="m:1"))
|
||||||
os._exit(1) # die without closing anything, as a power cut would
|
os._exit(1) # die without closing anything, as a power cut would
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@@ -48,5 +48,5 @@ def test_state_survives_process_death(tmp_path: Path) -> None:
|
|||||||
[pending] = store.pending_intents()
|
[pending] = store.pending_intents()
|
||||||
assert pending.payload == {"branch": "milestone-2"}
|
assert pending.payload == {"branch": "milestone-2"}
|
||||||
|
|
||||||
[stored] = store.read_events()
|
[stored] = store.all_events()
|
||||||
assert stored.event.text == "start milestone 2"
|
assert stored.event.text == "start milestone 2"
|
||||||
@@ -45,11 +45,13 @@ def build(
|
|||||||
def test_delegate_completes_a_scoped_subtask(
|
def test_delegate_completes_a_scoped_subtask(
|
||||||
tmp_path: Path, passthrough: AnthropicPassthroughClient, live_model: str
|
tmp_path: Path, passthrough: AnthropicPassthroughClient, live_model: str
|
||||||
) -> None:
|
) -> None:
|
||||||
delegation, repo, _store = build(
|
delegation, repo, store = build(
|
||||||
tmp_path, Caps(turns=12, wall_clock_s=300.0), passthrough, live_model
|
tmp_path, Caps(turns=12, wall_clock_s=300.0), passthrough, live_model
|
||||||
)
|
)
|
||||||
|
|
||||||
result = delegation.run(brief=BRIEF, branch="live/delegation-1")
|
result = delegation.run(
|
||||||
|
store.transcript("t", "helper"), brief=BRIEF, branch="live/delegation-1"
|
||||||
|
)
|
||||||
|
|
||||||
assert result.usable, f"delegation did not hand back: {result.status} {result.note}"
|
assert result.usable, f"delegation did not hand back: {result.status} {result.note}"
|
||||||
assert (repo.root / "slugify.py").exists()
|
assert (repo.root / "slugify.py").exists()
|
||||||
@@ -75,11 +77,13 @@ def test_a_cap_breach_discards_the_work(
|
|||||||
) -> None:
|
) -> None:
|
||||||
# One turn cannot finish the brief, so this exercises the abort path
|
# One turn cannot finish the brief, so this exercises the abort path
|
||||||
# against a real model rather than a scripted one.
|
# against a real model rather than a scripted one.
|
||||||
delegation, repo, _ = build(
|
delegation, repo, store = build(
|
||||||
tmp_path, Caps(turns=1, wall_clock_s=120.0), passthrough, live_model
|
tmp_path, Caps(turns=1, wall_clock_s=120.0), passthrough, live_model
|
||||||
)
|
)
|
||||||
|
|
||||||
result = delegation.run(brief=BRIEF, branch="live/delegation-2")
|
result = delegation.run(
|
||||||
|
store.transcript("t", "helper"), brief=BRIEF, branch="live/delegation-2"
|
||||||
|
)
|
||||||
|
|
||||||
assert not result.usable
|
assert not result.usable
|
||||||
assert result.status == "capped"
|
assert result.status == "capped"
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ def desk(tmp_path: Path, litellm: LiteLLMClient, live_model: str):
|
|||||||
store = StateStore(
|
store = StateStore(
|
||||||
tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO())
|
tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO())
|
||||||
)
|
)
|
||||||
return ExpertDesk(litellm, store, roles, budget=budget), store
|
return ExpertDesk(litellm, roles, budget=budget), store
|
||||||
|
|
||||||
return make
|
return make
|
||||||
|
|
||||||
@@ -35,8 +35,9 @@ def test_live_consult_parses_against_the_local_schema(desk) -> None:
|
|||||||
expert, store = desk()
|
expert, store = desk()
|
||||||
|
|
||||||
answer = expert.ask(
|
answer = expert.ask(
|
||||||
|
store.transcript("t", "expert"),
|
||||||
"For a single-process agent storing a few thousand rows of task state, "
|
"For a single-process agent storing a few thousand rows of task state, "
|
||||||
"is SQLite sufficient, or is PostgreSQL warranted? Answer briefly."
|
"is SQLite sufficient, or is PostgreSQL warranted? Answer briefly.",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert answer.confidence in {"high", "medium", "low"}
|
assert answer.confidence in {"high", "medium", "low"}
|
||||||
@@ -48,7 +49,10 @@ def test_live_consult_parses_against_the_local_schema(desk) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def test_live_budget_stops_a_circling_caller(desk) -> None:
|
def test_live_budget_stops_a_circling_caller(desk) -> None:
|
||||||
expert, _ = desk(budget=1)
|
expert, store = desk(budget=1)
|
||||||
expert.ask("Name one advantage of WAL mode in SQLite. Be brief.")
|
expert.ask(
|
||||||
|
store.transcript("t", "expert"),
|
||||||
|
"Name one advantage of WAL mode in SQLite. Be brief.",
|
||||||
|
)
|
||||||
with pytest.raises(BudgetExhausted):
|
with pytest.raises(BudgetExhausted):
|
||||||
expert.ask("And another?")
|
expert.ask(store.transcript("t", "expert"), "And another?")
|
||||||
@@ -72,10 +72,10 @@ def test_plan_approve_execute_produces_a_real_file(
|
|||||||
orchestrator = Orchestrator(
|
orchestrator = Orchestrator(
|
||||||
store,
|
store,
|
||||||
transport,
|
transport,
|
||||||
LLMPlanningAgent(client, store, roles),
|
LLMPlanningAgent(client, roles),
|
||||||
operators={"don"},
|
operators={"don"},
|
||||||
roles=roles,
|
roles=roles,
|
||||||
executor=LLMExecutor(client, store, roles, workspace, max_turns=10),
|
executor=LLMExecutor(client, roles, workspace, max_turns=10),
|
||||||
)
|
)
|
||||||
orchestrator.start()
|
orchestrator.start()
|
||||||
|
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ pytestmark = pytest.mark.integration
|
|||||||
def store_with(tmp_path: Path, *events) -> StateStore:
|
def store_with(tmp_path: Path, *events) -> StateStore:
|
||||||
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
for event in events:
|
for event in events:
|
||||||
store.append_event(event)
|
store.transcript("t", "helper").append(event)
|
||||||
return store
|
return store
|
||||||
|
|
||||||
|
|
||||||
@@ -35,7 +35,7 @@ def test_passthrough_completes_a_plain_turn(
|
|||||||
tmp_path, UserMessage(text="Reply with exactly: pong", source="live")
|
tmp_path, UserMessage(text="Reply with exactly: pong", source="live")
|
||||||
)
|
)
|
||||||
result = passthrough.complete(
|
result = passthrough.complete(
|
||||||
store.read_events(), None, live_model, system="Be terse."
|
store.transcript("t", "helper").read(), None, live_model, system="Be terse."
|
||||||
)
|
)
|
||||||
texts = [e.text for e in result if isinstance(e, AssistantText)]
|
texts = [e.text for e in result if isinstance(e, AssistantText)]
|
||||||
assert texts and "pong" in texts[0].lower()
|
assert texts and "pong" in texts[0].lower()
|
||||||
@@ -53,7 +53,7 @@ def test_passthrough_round_trips_a_tool_call(
|
|||||||
)
|
)
|
||||||
client = passthrough
|
client = passthrough
|
||||||
produced = client.complete(
|
produced = client.complete(
|
||||||
store.read_events(),
|
store.transcript("t", "helper").read(),
|
||||||
EXECUTOR_TOOLS,
|
EXECUTOR_TOOLS,
|
||||||
live_model,
|
live_model,
|
||||||
system="Use a tool. Do not answer in prose.",
|
system="Use a tool. Do not answer in prose.",
|
||||||
@@ -64,11 +64,13 @@ def test_passthrough_round_trips_a_tool_call(
|
|||||||
# Feed the result back: the follow-up render must be accepted, which is
|
# Feed the result back: the follow-up render must be accepted, which is
|
||||||
# what proves tool_result placement is right for this vendor.
|
# what proves tool_result placement is right for this vendor.
|
||||||
for event in produced:
|
for event in produced:
|
||||||
store.append_event(event)
|
store.transcript("t", "helper").append(event)
|
||||||
store.append_event(
|
store.transcript("t", "helper").append(
|
||||||
ToolResult(call_id=calls[0].call_id, result="app.py\ntests/", success=True)
|
ToolResult(call_id=calls[0].call_id, result="app.py\ntests/", success=True)
|
||||||
)
|
)
|
||||||
follow_up = client.complete(store.read_events(), EXECUTOR_TOOLS, live_model)
|
follow_up = client.complete(
|
||||||
|
store.transcript("t", "helper").read(), EXECUTOR_TOOLS, live_model
|
||||||
|
)
|
||||||
assert follow_up, "the vendor rejected or ignored the tool_result turn"
|
assert follow_up, "the vendor rejected or ignored the tool_result turn"
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ def test_live_planning_turn_returns_a_tool_action(
|
|||||||
model = live_model
|
model = live_model
|
||||||
roles = RoleMap(executor=model, planner=model, helper=model, expert=model)
|
roles = RoleMap(executor=model, planner=model, helper=model, expert=model)
|
||||||
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
store.append_event(
|
store.transcript("t", "operator").append(
|
||||||
UserMessage(
|
UserMessage(
|
||||||
text=(
|
text=(
|
||||||
"Project 'hello-cli': a Python script that prints a greeting. "
|
"Project 'hello-cli': a Python script that prints a greeting. "
|
||||||
@@ -36,12 +36,12 @@ def test_live_planning_turn_returns_a_tool_action(
|
|||||||
source="live-test",
|
source="live-test",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
agent = LLMPlanningAgent(litellm, store, roles)
|
agent = LLMPlanningAgent(litellm, roles)
|
||||||
|
|
||||||
action = agent.plan("arch", "hello-cli")
|
action = agent.plan(store.transcript("t", "planner"), "arch", "hello-cli")
|
||||||
|
|
||||||
assert isinstance(action, SubmitPlan | AskHuman)
|
assert isinstance(action, SubmitPlan | AskHuman)
|
||||||
text = action.content if isinstance(action, SubmitPlan) else action.question
|
text = action.content if isinstance(action, SubmitPlan) else action.question
|
||||||
assert text.strip(), "tool call carried empty content"
|
assert text.strip(), "tool call carried empty content"
|
||||||
# The turn must be durable regardless of what the model said (L8/L7).
|
# The turn must be durable regardless of what the model said (L8/L7).
|
||||||
assert len(store.read_events()) > 1
|
assert len(store.transcript("t", "planner").read()) > 1
|
||||||
Reference in new issue
Block a user