M4.1: toolchain preflight

A model works out and sets up what a project needs on its build VM;
code makes sure that happens, bounds it and checks the result (D3).

- Before each driver run on a project, one round trip checks the build
  VM: the harness's own baseline (missing: the item waits, a template
  problem), the workspace clone (missing: cloned again and said; the
  item resumes from what it pushed) and a per-project marker.
- No marker (a new project, a new or wiped VM): the item's driver model
  runs a bounded preflight first, with its own prompt, the execution
  tools (screened as ever) and toolchain_ready. The harness runs each
  reported version command itself (version-shaped commands only, even
  from the marker, which the model's commands could edit) and writes
  the marker. Gaps and an unfinished preflight are facts and work goes
  on; the thread hears only of installs and gaps. A rebuilt VM starts
  from the last toolchain.
- Later runs check the versions again; a change is a fact and a thread
  line. The PR body says what the work was tested with.
- A changed host key is its own failure (HostKeyChanged): the item
  waits for the operator's `trust new host key`, which drops the pin;
  then the VM is set up again.
- A prompt line: install what the work needs on the build VM, say why,
  and put lasting needs in the project's own pins.
- tuning.preflight: max_turns, command_timeout_s.
- The property walk gains a VM wipe and ready_before_work (checked
  against a mutant that skips preflight); parked_work_safe accepts that
  a wipe loses what wasn't pushed (D3).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE

nisse-dev/src/nisse_dev/loop/preflight.py

@@ -0,0 +1,282 @@
+"""Toolchain preflight (architecture D3; Development Plan M4.1).
+
+A model works out what a project needs and sets it up on the build VM; code
+makes sure that happens, bounds it and checks the result.
+
+- **Ready or not** is the harness's: before a driver run it checks, in one
+  round trip, the baseline it relies on itself, the workspace clone, and a
+  marker per project (``~/.nisse/<project>.ready``) that a finished preflight
+  leaves. No marker means a new project, VM or wipe: preflight runs first.
+- **The preflight run** is the item's driver model with its own prompt and
+  the execution tools (screened as ever), ending with ``toolchain_ready``.
+- **Versions are the harness's facts:** it runs each tool's version command
+  itself (version-shaped commands only) and records what they print; it
+  re-checks them before later runs, and a change is a fact for the driver.
+"""
+
+import base64
+import json
+import shlex
+from dataclasses import dataclass, field
+from typing import Any
+
+from nisse_core.actions import split_shell, unwrap
+from nisse_dev.execution import SSHHost
+
+# What the harness itself runs on a build VM (L13, D4): part of the
+# template, never of a project's toolchain.
+BASELINE = ("git", "base64", "realpath", "mkdir", "ssh-agent", "ssh-add", "timeout")
+
+_VERSION_FLAGS = ("--version", "-V", "-v", "version", "-version")
+
+
+@dataclass(frozen=True)
+class Readiness:
+    missing: tuple[str, ...]  # baseline commands the VM lacks
+    cloned: bool  # the workspace clone is there
+    marker: dict[str, Any] | None  # what the last preflight left, if any
+
+
+@dataclass(frozen=True)
+class Tool:
+    name: str
+    command: str  # version-shaped: prints the tool's version
+    version: str = ""  # what it printed, when the harness last ran it
+
+    def to_data(self) -> list[str]:
+        return [self.name, self.command, self.version]
+
+
+@dataclass
+class Report:
+    summary: str
+    tools: list[Tool] = field(default_factory=list)
+    missing: list[tuple[str, str]] = field(default_factory=list)  # (name, why)
+
+
+MARKERS = "$HOME/.nisse"  # on the build VM
+
+
+def marker_path(project: str, markers: str = MARKERS) -> str:
+    return f"{markers}/{project}.ready"
+
+
+def check(
+    host: SSHHost, project: str, workspace: str, markers: str = MARKERS
+) -> Readiness:
+    """The baseline, the clone and the marker, in one round trip."""
+    probe = "; ".join(
+        [
+            *(
+                f"command -v {shlex.quote(c)} >/dev/null 2>&1 || echo missing:{c}"
+                for c in BASELINE
+            ),
+            f"test -d {shlex.quote(workspace)}/.git && echo cloned",
+            (
+                f'[ -f "{marker_path(project, markers)}" ] && '
+                f'echo marker:$(base64 -w0 < "{marker_path(project, markers)}")'
+            ),
+            "true",
+        ]
+    )
+    result = host.execute(probe, timeout_s=30)
+    missing: list[str] = []
+    cloned = False
+    marker: dict[str, Any] | None = None
+    for line in result.stdout.splitlines():
+        if line.startswith("missing:"):
+            missing.append(line.removeprefix("missing:"))
+        elif line == "cloned":
+            cloned = True
+        elif line.startswith("marker:"):
+            try:
+                marker = json.loads(base64.b64decode(line.removeprefix("marker:")))
+            except ValueError:
+                marker = None  # unreadable: as good as none
+    return Readiness(tuple(missing), cloned, marker)
+
+
+def write_marker(
+    host: SSHHost, project: str, data: dict[str, Any], markers: str = MARKERS
+) -> None:
+    text = base64.b64encode(json.dumps(data, sort_keys=True).encode()).decode()
+    host.execute(
+        f'mkdir -p "{markers}" && base64 -d > "{marker_path(project, markers)}"',
+        stdin=text,
+        timeout_s=30,
+    )
+
+
+def version_shaped(command: str) -> bool:
+    """One plain command that only prints a version: ``rustc --version``,
+    ``go version``, ``python3 -V``. Nothing else is run by the harness."""
+    commands = split_shell(command)
+    if commands is None or len(commands) != 1:
+        return False
+    argv, written = commands[0]
+    if written or unwrap(list(argv)) != argv:
+        return False  # no redirection, no sudo or env in front
+    return (
+        2 <= len(argv) <= 3
+        and argv[-1] in _VERSION_FLAGS
+        and not argv[0].startswith(("-", "/", "."))
+        and argv[0] not in ("sh", "bash", "zsh", "dash", "eval", "exec")
+    )
+
+
+def versions(host: SSHHost, tools: list[Tool]) -> list[Tool]:
+    """Each tool's version, as its command prints it now (first line, from
+    stdout or stderr): run by the harness, in one round trip."""
+    # The marker lives on the VM, where the model's commands could change
+    # it: only version-shaped commands are ever run, wherever they came from.
+    tools = [t for t in tools if version_shaped(t.command)]
+    if not tools:
+        return []
+    script = "; ".join(
+        f"echo @@{i}; if command -v {shlex.quote(t.command.split()[0])} "
+        f">/dev/null 2>&1; then {t.command} 2>&1 | head -n 1; fi"
+        for i, t in enumerate(tools)
+    )
+    result = host.execute(script, timeout_s=60)
+    found: dict[int, str] = {}
+    current = -1
+    for line in result.stdout.splitlines():
+        if line.startswith("@@") and line[2:].isdigit():
+            current = int(line[2:])
+            found[current] = ""
+        elif current >= 0 and not found[current]:
+            found[current] = line.strip()
+    return [
+        Tool(t.name, t.command, found.get(i, "") or "not found")
+        for i, t in enumerate(tools)
+    ]
+
+
+def drift(before: list[Tool], now: list[Tool]) -> list[str]:
+    """What changed, as "name: old → new"."""
+    was = {t.name: t.version for t in before}
+    return [
+        f"{t.name}: {was[t.name]} → {t.version}"
+        for t in now
+        if t.name in was and was[t.name] != t.version
+    ]
+
+
+def tools_of(data: Any) -> list[Tool]:
+    """Tools from stored data (a marker's or an item's)."""
+    out: list[Tool] = []
+    for entry in data or []:
+        if isinstance(entry, list | tuple) and len(entry) == 3:
+            out.append(Tool(str(entry[0]), str(entry[1]), str(entry[2])))
+    return out
+
+
+def said(tools: list[Tool]) -> str:
+    return ", ".join(f"{t.name} {t.version}" for t in tools) or "nothing recorded"
+
+
+TOOLCHAIN_READY = {
+    "type": "function",
+    "function": {
+        "name": "toolchain_ready",
+        "description": (
+            "The project's toolchain is set up on this build VM (or as far as "
+            "it can be): report it. Ends the preflight."
+        ),
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "summary": {
+                    "type": "string",
+                    "description": "What you found and did, in a sentence or two.",
+                },
+                "toolchain": {
+                    "type": "array",
+                    "description": (
+                        "Each tool the project needs whose version matters, "
+                        "with the command that prints its version (e.g. "
+                        "'python3 --version', 'rustc -V', 'go version')."
+                    ),
+                    "items": {
+                        "type": "object",
+                        "properties": {
+                            "name": {"type": "string"},
+                            "version_command": {"type": "string"},
+                        },
+                        "required": ["name", "version_command"],
+                    },
+                },
+                "installed": {
+                    "type": "array",
+                    "items": {"type": "string"},
+                    "description": "What you installed, if anything.",
+                },
+                "missing": {
+                    "type": "array",
+                    "description": "What the project needs that you couldn't set up.",
+                    "items": {
+                        "type": "object",
+                        "properties": {
+                            "name": {"type": "string"},
+                            "why": {"type": "string"},
+                        },
+                        "required": ["name", "why"],
+                    },
+                },
+            },
+            "required": ["summary", "toolchain"],
+        },
+    },
+}
+
+SYSTEM = """You are nisse-dev, getting a project's toolchain ready on its build VM
+before any work starts. The project is {repo}; its clone is the workspace.
+{workspace}
+
+Read the project's own files to work out what it needs to build, test and run:
+its language runtimes and compilers, package and build tools, and anything else
+it calls (system libraries, CLIs, a database for its tests). Honour its pins
+(.python-version, rust-toolchain.toml, requires-python, engines, lockfiles).
+{last}
+Check what's there with version commands; install what's missing. The build
+VM is the project's and disposable: installing on it is fine. Don't install or
+change anything for other hosts, and never install, store or ask for
+credentials: the harness hands those out when they're needed.
+
+Leave out the basics every machine has (ls, cat, git, curl, ssh, tar): those
+are the VM template's. Leave out tools that act on other systems (gh, tea,
+docker push, cloud CLIs): they aren't how work is done here.
+
+When you're done, or can't get further, call toolchain_ready with what you
+found and did: each tool whose version matters with a command that prints its
+version, what you installed, and what's still missing and why. Keep it short:
+this is setup, not the work."""
+
+
+def report_of(arguments: dict[str, Any]) -> Report:
+    tools = [
+        Tool(str(t.get("name", "")).strip(), str(t.get("version_command", "")).strip())
+        for t in arguments.get("toolchain") or []
+        if isinstance(t, dict)
+    ]
+    missing = [
+        (str(m.get("name", "")).strip(), str(m.get("why", "")).strip())
+        for m in arguments.get("missing") or []
+        if isinstance(m, dict)
+    ]
+    return Report(str(arguments.get("summary", "")).strip(), tools, missing)
+
+
+def report_problem(arguments: dict[str, Any]) -> str | None:
+    """A toolchain_ready the harness can't use: said back to the model."""
+    report = report_of(arguments)
+    bad = [t.command for t in report.tools if not version_shaped(t.command)]
+    if bad:
+        return (
+            "these aren't plain version commands, which is all I'll run: "
+            + "; ".join(bad)
+            + ". Use the tool's own --version (or -V, version)."
+        )
+    if any(not t.name for t in report.tools):
+        return "every toolchain entry needs a name"
+    return None