Add ask_expert and eval capture (A14/A16) — step 7c

Completes step 7.

- expert.py: one stateless call for a question needing no repo access.
  Structured output is contracted at the prompt level, since vendors
  differ on JSON enforcement, so the reply is parsed and validated
  locally with one retry — tolerating a fenced code block, because
  models wrap JSON habitually, but not missing fields
- The per-task budget is counted from the canonical log rather than
  memory, so it survives a restart and cannot hand a circling executor
  a fresh allowance. Exhaustion raises rather than answering: a caller
  out of consults should get on with the work or escalate
- Consults are recorded as delegation_started/completed with role
  consultant. A16 asks for a "consult/delegation eval set", and
  reusing the composite events gets consult evaluation for free rather
  than inventing a parallel record
- evals.py pairs starts with completions into eval records, with
  verdicts kept outside the canonical log: the log records what
  happened, a verdict is a later judgement about it, and judgements
  get revised. Schema v3 adds the outcomes table, so a v1 store now
  traverses two migrations — a better exercise of the hook than one
- ask_expert is offered to the executor as a tool only when a desk is
  configured, in addition to the observable escalation triggers and
  never instead of them (A12). Consulting is not mutating, so a
  completion claim backed only by consults is still challenged

Found live: passing an empty message list and putting the question in
the system prompt fails outright — the Messages API requires at least
one non-system message. The question now travels as a synthesised user
message that is never appended to the log, which keeps the consult
stateless while remaining a valid request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

nisse-dev/src/nisse_dev/executor.py

@@ -11,12 +11,13 @@ the human with the failures attached rather than engaging a frontier model.
 """
 
 from dataclasses import dataclass
-from typing import Literal
+from typing import Any, Literal
 
 from nisse_dev.canonical import ToolCall, ToolResult
+from nisse_dev.expert import ASK_EXPERT_TOOL, BudgetExhausted, ExpertDesk
 from nisse_dev.llm import LLMClient, LLMError, RoleMap
 from nisse_dev.state import StateStore
-from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
+from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, ToolOutcome, Workspace
 
 EXECUTOR_SYSTEM = """You are nisse-dev's executor, doing mechanical work.
 
@@ -56,6 +57,7 @@ class LLMExecutor:
         *,
         max_turns: int = DEFAULT_MAX_TURNS,
         failure_limit: int = DEFAULT_FAILURE_LIMIT,
+        expert: ExpertDesk | None = None,
     ) -> None:
         self._client = client
         self._store = store
@@ -63,6 +65,28 @@ class LLMExecutor:
         self._workspace = workspace
         self._max_turns = max_turns
         self._failure_limit = failure_limit
+        self._expert = expert
+
+    def _tools(self) -> list[dict[str, Any]]:
+        # A12: ask_expert exists in addition to the observable escalation
+        # triggers, never as a replacement for them.
+        if self._expert is None:
+            return EXECUTOR_TOOLS
+        return [*EXECUTOR_TOOLS, ASK_EXPERT_TOOL]
+
+    def _consult(self, arguments: dict[str, Any]) -> ToolOutcome:
+        assert self._expert is not None
+        question = str(arguments.get("question", "")).strip()
+        if not question:
+            return ToolOutcome("ask_expert needs a question", success=False)
+        try:
+            return ToolOutcome(self._expert.ask(question).render(), success=True)
+        except BudgetExhausted as exc:
+            # Not an error the model should route around: it is the signal
+            # that it is circling and should get on with the work or fail.
+            return ToolOutcome(str(exc), success=False)
+        except LLMError as exc:
+            return ToolOutcome(f"the consultant did not answer: {exc}", success=False)
 
     def run(self, milestone: int) -> ExecutorOutcome:
         model = self._roles.resolve("executor")
@@ -78,7 +102,7 @@ class LLMExecutor:
             try:
                 produced = self._client.complete(
                     self._store.read_events(),
-                    tools=EXECUTOR_TOOLS,
+                    tools=self._tools(),
                     model=model,
                     system=system,
                 )
@@ -110,7 +134,10 @@ class LLMExecutor:
             for call in calls:
                 if call.tool in MUTATING_TOOLS:
                     did_work = True
-                outcome = self._workspace.dispatch(call.tool, call.arguments)
+                if call.tool == "ask_expert":
+                    outcome = self._consult(call.arguments)
+                else:
+                    outcome = self._workspace.dispatch(call.tool, call.arguments)
                 self._store.append_event(
                     ToolResult(
                         call_id=call.call_id,