M4.4: an unreachable build VM blocks its projects until it's back (#137)
M4.3's block, with new triggers and a way out. A VM out of reach at the check before a run, in a run (which ends there: the driver, a toolchain step and a requested step), or while preparing the workspace blocks every project on it; the item stays ready, waiting on no one. On the intake round it is tried again every tuning.build_vm.retry_s (300), and once it answers it must still answer settle_s (180) later; then the work a lost run left there is parked (pushed), the VM unblocked, and that said once. A changed host key, at the check or on a retry, blocks the VM too, until the operator's `vm ready`, which trusts the new key: `trust new host key` is retired, and the message says it's a key problem that may be an impersonation. Questions are answered while a VM is blocked, without it. Starting the agent with a VM down no longer fails. The walk breaks the VM's SSH, now or partway through a run, and checks that no model works on a VM that is down; loop.qnt models the VM going and coming back (unblocked_only_when_up). Together they found: a lost run's work set aside by the next item there; a question left behind a block; a lost VM swallowed while the toolchain step checked or recorded it; and a remark waiting to be answered that outlived its PR's merge. All fixed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
This commit is contained in:
1 parent
1091178a81
commit
a921b73d34
19 files changed
+935
-158
No files matched your search
@@ -78,10 +78,8 @@ class Loop:
|
||||
# it's said once per commit and reason.
|
||||
merge_held: str = ""
|
||||
# The toolchain as the harness last checked it (M4.1): name, version
|
||||
# command, version; and a build VM whose host key changed, waiting on
|
||||
# the operator's word.
|
||||
# command, version.
|
||||
toolchain: tuple[tuple[str, str, str], ...] = ()
|
||||
host_key_changed: bool = False
|
||||
|
||||
def to_data(self) -> dict[str, Any]:
|
||||
return asdict(self)
|
||||
@@ -89,6 +87,8 @@ class Loop:
|
||||
|
||||
def loop_of(item: WorkItem) -> Loop:
|
||||
data = dict(item.data.get("loop", {}))
|
||||
# Retired: a changed host key now blocks the build VM (#137).
|
||||
data.pop("host_key_changed", None)
|
||||
data["notes"] = tuple(data.get("notes", ()))
|
||||
data["blocked_by"] = tuple(data.get("blocked_by", ()))
|
||||
data["labels"] = tuple(data.get("labels", ()))
|
||||
|
||||
Reference in new issue
Block a user