M4.3: a broken build VM blocks its projects until it's ready #138

2026-10-08T19:27:28Z

dev-agent commented

2026-10-08 19:27:28 +00:00

M4.3, revised with the operator: a build VM that the driver reports broken blocks every project on it until the operator says it's ready. The unreachable case is #137, which reuses the same block.

What changes

request_reimage(reason), a new driver tool that ends the run. It's offered before the go-ahead too, because it asks rather than acts. The prompt says to try the usual fixes first (reinstall dependencies, toolchain_step), then ask rather than keep diagnosing.
Work in progress is pushed first. The harness parks the workspace, which pushes to Gitea as a project switch does. If the VM is too broken to push, the message says the work will be lost.
A durable block per build host (state schema 5, vm_blocks, with a migration from 4). Items on any project using that VM are never picked while it's blocked, across restarts, and status says why.
Messages:
- one top-level message giving the host, the reason and the projects blocked, ending with the exact command to send: "Once it's ready, say vm ready <project>; that also trusts its new host key";
- one line in each blocked item's thread.
vm ready [<project>] lifts the block and trusts the new host key. The project name can be left out when only one VM is blocked.
- A reimaged VM has no marker, so the toolchain step sets it up again from the repository before work.
- A VM said ready without a reimage goes straight on.
- The item that found the problem goes first.
A changed VM address still means editing build_host and restarting, and the message says so. That's deferred to M6.2 and M7.6.

Model and walk

loop.qnt has the block, the property blocked_vm_runs_nothing, a reachability witness and a scenario.
Simulation found a real gap: a reimage threw away the item's unpushed work. That's why the harness now parks first.
After the fix: 200,000 traces pass. A mutant without the guard fails blocked_vm_runs_nothing.
The code walk now requests reimages and recovers from them, with and without an actual wipe. It checks that no model call is made for an item while its VM is blocked. All 48 seeds pass, with 19 vm ready steps reached.

Tests

test_loop_toolchain.py:
- a broken VM blocks its project, survives a restart, is cleared by vm ready, and is set up again;
- ready without a reimage goes straight on;
- work in progress survives a reimage. This test fails without the park.
- vm ready with nothing blocked says so.
test_state.py: a block persists across a restart, and a version-4 store migrates.
Locally: the unit tier (685 tests) and the 48-seed walk pass. CI runs the rest.

Docs

Architecture P5 now says "fixed the usual ways, replaced when that doesn't work".
D3 describes the block.
The M4.3 entry in the Development Plan is rewritten, and #137 is added to the open-issues table.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

M4.3, revised with the operator: a build VM that the driver reports broken blocks every project on it until the operator says it's ready. The unreachable case is #137, which reuses the same block. ## What changes - **`request_reimage(reason)`**, a new driver tool that ends the run. It's offered before the go-ahead too, because it asks rather than acts. The prompt says to try the usual fixes first (reinstall dependencies, `toolchain_step`), then ask rather than keep diagnosing. - **Work in progress is pushed first.** The harness parks the workspace, which pushes to Gitea as a project switch does. If the VM is too broken to push, the message says the work will be lost. - **A durable block per build host** (state schema 5, `vm_blocks`, with a migration from 4). Items on any project using that VM are never picked while it's blocked, across restarts, and `status` says why. - **Messages:** - one top-level message giving the host, the reason and the projects blocked, ending with the exact command to send: "Once it's ready, say `vm ready <project>`; that also trusts its new host key"; - one line in each blocked item's thread. - **`vm ready [<project>]`** lifts the block and trusts the new host key. The project name can be left out when only one VM is blocked. - A reimaged VM has no marker, so the toolchain step sets it up again from the repository before work. - A VM said ready without a reimage goes straight on. - The item that found the problem goes first. - **A changed VM address** still means editing `build_host` and restarting, and the message says so. That's deferred to M6.2 and M7.6. ## Model and walk - `loop.qnt` has the block, the property `blocked_vm_runs_nothing`, a reachability witness and a scenario. - Simulation found a real gap: a reimage threw away the item's unpushed work. That's why the harness now parks first. - After the fix: 200,000 traces pass. A mutant without the guard fails `blocked_vm_runs_nothing`. - The code walk now requests reimages and recovers from them, with and without an actual wipe. It checks that no model call is made for an item while its VM is blocked. All 48 seeds pass, with 19 `vm ready` steps reached. ## Tests - `test_loop_toolchain.py`: - a broken VM blocks its project, survives a restart, is cleared by `vm ready`, and is set up again; - ready without a reimage goes straight on; - work in progress survives a reimage. This test fails without the park. - `vm ready` with nothing blocked says so. - `test_state.py`: a block persists across a restart, and a version-4 store migrates. - Locally: the unit tier (685 tests) and the 48-seed walk pass. CI runs the rest. ## Docs - Architecture P5 now says "fixed the usual ways, replaced when that doesn't work". - D3 describes the block. - The M4.3 entry in the Development Plan is rewritten, and #137 is added to the open-issues table. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

dev-agent added 1 commit 2026-10-08 19:27:28 +00:00

M4.3: a broken build VM blocks its projects until it's ready

CI / checks (push) Skipped

CI / checks (pull_request) Successful in 1m38s

Details

CI / e2e (pull_request) Successful in 4m20s

Details

c794a1e52b

The driver can request_reimage(reason) when its build VM is broken past the
usual fixes (before the go-ahead too: it asks, it doesn't act). The harness
parks the work in progress (pushed to Gitea), then blocks the VM in the state
store (schema 5, vm_blocks): no item on any project on it is taken up, across
restarts. The operator hears once at top level, with the command to clear it,
and each blocked item's thread gets a line. `vm ready <project>` lifts the
block and trusts the VM's new host key; a reimaged VM is set up again by the
toolchain step, and the item that found it goes first.

loop.qnt gains the block (blocked_vm_runs_nothing) and a scenario; it found
that a reimage lost work in progress, hence parking first. The walk requests
reimages and recovers them, reimaged or not. The unreachable case reuses the
block in #137. Architecture P5 is softer, and D3 describes the block.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

dongar approved these changes

2026-10-08 19:40:48 +00:00

dev-agent merged commit 1091178a81 into main

2026-10-08 19:41:06 +00:00

dev-agent deleted branch environment-repair

2026-10-08 19:41:06 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/nisse#138