Run every command on the build VM over SSH (M1.3)
CI / checks (push) Skipped
CI / checks (pull_request) Failing after 55s

execution.SSHHost.execute is now the only way the agent runs anything
(L13). Commands travel as a script on SSH's stdin, so the secrets handed
to a call are never on a command line, and every value handed out is
redacted from the output. Timeouts kill the remote process group, and
an unreachable host is reported as such, not as a failed command.

Workspace and GitRepo run on the build VM. The clone uses the
repository's SSH URL, so no credential is stored in it (#89); fetches
and pushes get the push key over stdin, in a short-lived ssh-agent,
and PRs are still opened from the agent's host with its API token.
The bench runs on the build host too.

Tests reach "the build VM" through a throwaway local sshd (nisse_dev
.testing); CI installs openssh-server. The live push test now pushes
with the sandbox's deploy key (NISSE_PUSH_KEY).

Fixes #89.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-10-02 21:39:46 +00:00
1 parent b7f1f3371c
commit 761ebdb857
32 files changed
+1098 -280

No files matched your search

+25
View File
@@ -13,13 +13,18 @@ Each task is a directory under ``tasks/``:
"""
import hashlib
import shlex
import shutil
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
import yaml
if TYPE_CHECKING:
from nisse_dev.tools import Workspace
TASKS_DIR = Path(__file__).parent / "tasks"
CHECK_DIR = ".bench_check"
@@ -75,6 +80,26 @@ class Task:
output = (result.stdout + result.stderr)[-2000:]
return result.returncode == 0, output
# -- on the build VM, where the agent's runs happen (L13) ---------------
def seed_workspace(self, workspace: "Workspace") -> None:
workspace.upload_tree(self.path / "seed")
def check_workspace(
self, workspace: "Workspace", timeout_s: int = 300
) -> tuple[bool, str]:
"""run_check, on the build VM: copy the hidden files in fresh (never
trusting a directory the model made) and run the check there."""
root = shlex.quote(str(workspace.root / CHECK_DIR))
workspace.host.execute(f"rm -rf -- {root}", timeout_s=60)
workspace.upload_tree(self.path / "hidden", CHECK_DIR)
result = workspace.host.execute(
self.check, cwd=str(workspace.root), timeout_s=timeout_s
)
if result.timed_out:
return False, f"check timed out after {timeout_s}s"
return result.exit_code == 0, (result.stdout + result.stderr)[-2000:]
def load_task(path: Path) -> Task:
spec = yaml.safe_load((path / "task.yaml").read_text())