M4.4: an unreachable build VM blocks its projects until it's back (#137)

M4.3's block, with new triggers and a way out. A VM out of reach at the
check before a run, in a run (which ends there: the driver, a toolchain
step and a requested step), or while preparing the workspace blocks every
project on it; the item stays ready, waiting on no one. On the intake round
it is tried again every tuning.build_vm.retry_s (300), and once it answers
it must still answer settle_s (180) later; then the work a lost run left
there is parked (pushed), the VM unblocked, and that said once.

A changed host key, at the check or on a retry, blocks the VM too, until
the operator's `vm ready`, which trusts the new key: `trust new host key`
is retired, and the message says it's a key problem that may be an
impersonation. Questions are answered while a VM is blocked, without it.
Starting the agent with a VM down no longer fails.

The walk breaks the VM's SSH, now or partway through a run, and checks that
no model works on a VM that is down; loop.qnt models the VM going and
coming back (unblocked_only_when_up). Together they found: a lost run's
work set aside by the next item there; a question left behind a block; a
lost VM swallowed while the toolchain step checked or recorded it; and a
remark waiting to be answered that outlived its PR's merge. All fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

nisse-dev/src/nisse_dev/tools.py

@@ -93,6 +93,7 @@ EXECUTION_TOOLS: list[dict[str, Any]] = [
 class ToolOutcome:
     result: str
     success: bool
+    unreachable: bool = False  # the build VM couldn't be reached (#137)
 
 
 class Workspace:
@@ -118,7 +119,12 @@ class Workspace:
         # The project secrets the model's commands may see (D7); redacted from
         # whatever comes back.
         self._secrets = dict(secrets or {})
-        self._run(f"mkdir -p -- {shlex.quote(str(self.root))}")
+        try:
+            self._run(f"mkdir -p -- {shlex.quote(str(self.root))}")
+        except HostUnavailable:
+            # A VM down at startup mustn't stop the agent: the check before
+            # each run finds it, and blocks it (#137); cloning makes the root.
+            pass
 
     def with_timeout(self, timeout_s: int) -> "Workspace":
         """The same workspace with a longer limit per command (installs)."""
@@ -173,7 +179,9 @@ class Workspace:
             return ToolOutcome(f"missing argument {exc} for {tool}", success=False)
         except HostUnavailable as exc:
             # Not the work failing: the build VM is out of reach.
-            return ToolOutcome(f"the build VM is unreachable: {exc}", success=False)
+            return ToolOutcome(
+                f"the build VM is unreachable: {exc}", success=False, unreachable=True
+            )
 
     def read_file(self, path: str) -> ToolOutcome:
         target = self._resolve(path)
@@ -244,7 +252,9 @@ class Workspace:
                 timeout_s=self._timeout_s,
             )
         except HostUnavailable as exc:
-            return ToolOutcome(f"the build VM is unreachable: {exc}", success=False)
+            return ToolOutcome(
+                f"the build VM is unreachable: {exc}", success=False, unreachable=True
+            )
         if result.timed_out:
             return ToolOutcome(
                 f"command timed out after {self._timeout_s}s: {command}",