Conversation on the new loop (M2.7)

- Questions never change state (L5, #77): a message ending in "?" to a
  waiting project item starts a question run (reply and look-only tools,
  an instruction to change nothing), and the item goes back to the very
  wait it was in: a restatement up stays up and a go binds to it; the
  agent's own question stays open.
- Explanations: a restatement gives its reasons and the main alternative;
  a revision says what changed (restate(changed), sent back without it);
  a milestone's summary says what was done, verified and left open.
- Progress in prose (#42): the driver's and a helper's own words for a
  turn are posted, trimmed, at most once a minute per item; `details`
  shows the last tool calls; a message arriving mid-run is acknowledged at
  once ("Noted; ...").
- CI: the e2e job's comments and the plan no longer claim a newer push
  cancels the run it supersedes (Gitea here doesn't); the concurrency
  block goes.

Tests: conversation unit tests (questions in a go-ahead and an answer wait,
an unanswerable question, revisions, progress rate and trimming, details,
mid-run notes), the helper's progress, the walk asking questions in every
state with a questions_change_nothing property (routing questions the old
way fails 32 of 48 seeds), and an e2e question while a restatement is up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE

nisse-dev/src/nisse_dev/loop/agent.py

@@ -29,6 +29,8 @@ from nisse_dev.canonical import (
     DelegationStarted,
     Fact,
     PlanSubmitted,
+    ToolCall,
+    ToolResult,
     UserMessage,
 )
 from nisse_dev.config import GENERAL
@@ -87,6 +89,9 @@ _NAMED = re.compile(r"(go|approve|abandon|stop) (?:([a-z0-9][a-z0-9-]*))?#(\d+)"
 _NEEDS_ITEM = frozenset({"go", "approve", "abandon"})
 BUG_GRACE_S = 600.0
 GENERAL_IDLE_S = 86400.0
+PROGRESS_EVERY_S = 60.0  # progress in prose at a readable rate (L5, #42)
+PROGRESS_CHARS = 400
+DETAILS = 10  # tool calls shown by `details`
 FILE_ISSUE = "file_issue"  # the intent of opening a record issue (L8)
 # Carried by a record issue the agent opens for work started in conversation:
 # intake never takes it for new work, even before its item exists (D1).
@@ -102,9 +107,14 @@ The issue and the conversation about it are below.
 
 Before any work, find out what you need, then use restate to put your
 understanding to the operator: what you take the request to mean, how it fits
-the project, and your first step. Wait for their go-ahead. If the request is
-unclear, ask_human instead of guessing: a restatement asks no questions, so
-the operator can agree to it as it stands.
+the project, your first step, and why that way rather than the main
+alternative. A later restatement says what changed. Wait for their go-ahead.
+If the request is unclear, ask_human instead of guessing: a restatement asks
+no questions, so the operator can agree to it as it stands. At any point,
+asking is better than guessing.
+
+Before each group of tool calls, say in a sentence what you're doing and why:
+the operator sees it as your progress.
 
 The project's documents are in its repository: read ARCHITECTURE.md and any
 build plans under docs/plans/ before you restate.
@@ -119,7 +129,8 @@ the same milestone (don't start another), run the tests, and call
 finish_milestone again to push the fix. When a review needs no change from
 you, or you are only waiting for the operator to merge (you can't merge it
 yourself), call wait_for_review. The next milestone starts only after the
-previous PR has merged. When the last PR has merged, call close_item with
+previous PR has merged. A milestone's summary says what was done, what was
+verified, and what is left open. When the last PR has merged, call close_item with
 what was done, what was verified, and what is left open. If a PR is closed
 without merging, ask the operator what next.
 
@@ -133,6 +144,12 @@ Keep ARCHITECTURE.md current: a design change updates it in the same PR.
 
 {workspace}"""
 
+QUESTION = """
+
+The operator has asked a question (the last message). Answer it with reply,
+from what you know and what you can look at; don't change anything. The item
+then goes back to waiting exactly as it was, so a restatement up stays up."""
+
 CONVERSATION = """You are nisse-dev, a software agent, in a conversation with
 the operator in a Matrix thread. It isn't about a project yet: it may be a
 question, thoughts on feasibility or hardware, or the start of some work.
@@ -257,6 +274,9 @@ class LoopAgent:
         # The project whose workspace was last worked on: switching to
         # another parks it (L1). None after a boot: unknown, so all are.
         self._last_project: str | None = None
+        # Per item: when progress was last posted (L5: a readable rate).
+        self._progressed: dict[str, float] = {}
+        self._active_name = ""  # the running item, for "noted" on arrival
         self._running = False
         self._stop = threading.Event()
         self._intake_thread: threading.Thread | None = None
@@ -743,6 +763,16 @@ class LoopAgent:
             # Straight to the flag: the run is what it has to reach (L2).
             self._cancel.set()
         about = self._shown.get(thread) if thread else None
+        if self._running and command not in ("stop", "status"):
+            # L5: say what happens to a message that can't be read yet.
+            here = thread and thread == self._active_thread
+            self._transport.post(
+                "Noted; I'll read it when this turn ends."
+                if here
+                else f"Noted; I'm in the middle of {self._active_name}, and I'll "
+                "read it when that turn ends.",
+                thread=thread or msg.event_id or None,
+            )
         self._worker.submit(
             self._settled("your message", lambda: self.handle(msg, about))
         )
@@ -814,6 +844,30 @@ class LoopAgent:
             self._close(item, loop, "abandoned", f"abandoned by {msg.sender}")
         elif command == "done" and loop.new_project:
             self._provisioned(item, loop)
+        elif command == "details":
+            self._transport.post(self._details(item), thread=loop.thread)
+        elif (
+            item.position == WAITING
+            and item.scope != GENERAL
+            and is_question(text)
+            and loop.wait in (GO_AHEAD, ANSWER, MERGE)
+        ):
+            # L5: questions never change state. Answered, then back to this
+            # very wait: a restatement up stays up, a go still binds to it.
+            save(
+                self._store,
+                item.id,
+                READY,
+                replace(
+                    loop,
+                    question=True,
+                    back_to=loop.wait,
+                    back_note=loop.note,
+                    wait="",
+                    note="",
+                ),
+                cause="the operator asked a question",
+            )
         elif command == "stop":
             # Stopping a run took effect on arrival; a stop is never an answer.
             if not loop.note.startswith("stopped ("):
@@ -1332,6 +1386,9 @@ class LoopAgent:
             caps=self._helpers.caps,
             cancel=self._cancel,
             reader=self._helpers.reader(roles),
+            on_turn=lambda said: self._progress(
+                f"{item.id}:helper", loop.thread, said, prefix="helper: "
+            ),
         )
         git = milestones.git
         before = git.head_sha()  # where the milestone is, uncommitted work aside
@@ -1432,6 +1489,54 @@ class LoopAgent:
         save(self._store, item.id, READY, loop_of(item), cause="a helper finished")
         return True
 
+    def _changed_problem(self, item_id: str, args: dict[str, object]) -> str | None:
+        """L5: a revision says what changed from the one before."""
+        item = self._store.work_item(item_id)
+        if item is None or loop_of(item).rev == 0:
+            return None
+        if str(args.get("changed", "")).strip():
+            return None
+        return (
+            f"not posted: this is a revision of rev-{loop_of(item).rev}, so say "
+            "in changed what's different and why"
+        )
+
+    def _progress(self, item_id: str, thread: str, said: str, prefix: str = "") -> None:
+        """A turn's own words, posted at a readable rate (L5, #42): at most
+        one a minute per item, the latest when the minute is up."""
+        now = self._clock()
+        if now - self._progressed.get(item_id, float("-inf")) < PROGRESS_EVERY_S:
+            return
+        self._progressed[item_id] = now
+        text = said if len(said) <= PROGRESS_CHARS else said[:PROGRESS_CHARS] + " …"
+        self._transport.post(f"{prefix}{text}", thread=thread)
+
+    def _details(self, item: WorkItem) -> str:
+        """L5: raw tool activity, on request: the item's last tool calls."""
+        events = self._store.transcript(item.id, "driver").read()
+        results = {
+            e.event.call_id: e.event for e in events if isinstance(e.event, ToolResult)
+        }
+        lines = []
+        for e in events:
+            if not isinstance(e.event, ToolCall):
+                continue
+            call, result = e.event, results.get(e.event.call_id)
+            target = call.arguments.get("path") or call.arguments.get("command") or ""
+            outcome = (
+                "no result yet"
+                if result is None
+                else ("ok" if result.success else "failed")
+                + (f": {result.result.strip()[:120]}" if result.result.strip() else "")
+            )
+            who = "" if e.actor == "driver" else f"{e.actor}: "
+            lines.append(f"- {who}{call.tool}({str(target)[:80]}) → {outcome}")
+        if not lines:
+            return f"No tool calls yet on {self._label(item)}."
+        return f"The last tool calls on {self._label(item)}:\n" + "\n".join(
+            lines[-DETAILS:]
+        )
+
     def _design_problem(self, item_id: str) -> str | None:
         """D1, D2: a `design` item agrees its design before any code. Until
         a PR of it has merged, a milestone holds documents only."""
@@ -1544,6 +1649,8 @@ class LoopAgent:
         save(self._store, item.id, RUNNING, loop, cause="driver run")
         # D1: a `discuss` item talks before it restates.
         hold = "discuss" in loop.labels and not loop.talked and not loop.started
+        if loop.question:
+            hold = True  # nothing but answering (below)
         tools = [
             *([] if hold else [RESTATE]),
             ASK_HUMAN,
@@ -1560,6 +1667,8 @@ class LoopAgent:
                 ]
             ),
         ]
+        if loop.question:
+            tools = [REPLY, *_LOOK]
         roles = self._roles_of(item)
         workspace = project.workspace
         system = SYSTEM.format(
@@ -1573,8 +1682,9 @@ class LoopAgent:
                 else ""
             ),
             workspace=workspace.describe() if workspace else "",
-        )
+        ) + (QUESTION if loop.question else "")
         self._active_thread = loop.thread
+        self._active_name = self._label(item)
         self._running = True
         try:
             outcome = self._driver.run(
@@ -1592,10 +1702,14 @@ class LoopAgent:
                     ),
                 },
                 checks={
-                    "restate": _restatement_problem,
+                    "restate": lambda args: (
+                        _restatement_problem(args)
+                        or self._changed_problem(item.id, args)
+                    ),
                     "finish_milestone": lambda args: self._design_problem(item.id),
                 },
                 model=roles.resolve("driver"),
+                on_turn=lambda said: self._progress(item.id, loop.thread, said),
             )
         finally:
             self._running = False
@@ -1607,6 +1721,31 @@ class LoopAgent:
         assert item is not None
         loop = loop_of(item)
         args = outcome.arguments or {}
+        if loop.question:
+            # Back to the wait the question interrupted, as it was (L5).
+            if outcome.kind == "wait" and outcome.tool == "reply":
+                self._transport.post(
+                    str(args.get("text", "")).strip(), thread=loop.thread
+                )
+            else:
+                self._transport.post(
+                    f"I couldn't answer that ({outcome.note or outcome.kind}).",
+                    thread=loop.thread,
+                )
+            self._wait(
+                item,
+                replace(
+                    loop,
+                    wait=loop.back_to,
+                    note=loop.back_note,
+                    question=False,
+                    back_to="",
+                    back_note="",
+                ),
+                None,
+            )
+            self._store.set_active(None, cause="question answered")
+            return
         if outcome.kind == "wait" and outcome.tool == "restate":
             text = str(args.get("text", "")).strip()
             rev = loop.rev + 1
@@ -1622,7 +1761,11 @@ class LoopAgent:
                     f"It's a bug, so I'll go ahead in {round(self._grace / 60)} min "
                     "unless you reply; say go to start now, or reply with changes."
                 )
-            self._transport.post(f"rev-{rev}: {text}\n\n{after}", thread=loop.thread)
+            changed = str(args.get("changed", "")).strip()
+            since = f"\n\nChanged from rev-{rev - 1}: {changed}" if rev > 1 else ""
+            self._transport.post(
+                f"rev-{rev}: {text}{since}\n\n{after}", thread=loop.thread
+            )
             self._wait(
                 item,
                 replace(loop, rev=rev, wait=GO_AHEAD, note="", grace_until=grace),