M2.8: switch over to the new loop
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m16s
CI / e2e (pull_request) Successful in 10m51s

nisse-dev now runs only the conversational loop. The gated state machine
it was built alongside is gone:

- Removed control.py, planner.py, executor.py, approvals.py, gates.py
  (command_form moves to loop/reader.py), publish.py and poller.py, the
  old assembly in agent.py (its shared helpers move to loop/build.py),
  the store's state-machine view (get_state, transition), their tests,
  the old end-to-end and live planning/milestone/gate tests, and
  specs/orchestrator.qnt with its scenarios.
- Roles are driver, helper, expert and classifier; driver is required.
  loop:, roles.planner, roles.executor and tuning.executor are refused
  with what to use instead; tuning.driver takes the turn cap and failure
  limit.
- The benchmark and the capability matrix measure the driver: the
  loop's own system prompt and tools, the task's plan as the agreed
  restatement. Task plans end with finish_milestone, so earlier runs
  show as (old); stored runs' executor tuning reads as the driver's.
- A delegation on the loop records what its helper's alias resolved to
  (L9), as the old path's did.
- The dump shows a work item's revision and what it waits for.
- Behaviour the old tests covered and the loop shares stays covered:
  driver runs that fail on a model call, stop mid-turn, reset the
  failure count; go-aheads ignoring case and punctuation; a request too
  big to fit is never sent; the assembly tests, now on build_loop_agent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-10-08 03:12:58 +00:00
1 parent 09232dcbdd
commit 6ae2375194
89 files changed
+951 -8592

No files matched your search

+28 -18
View File
@@ -24,8 +24,8 @@ Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
| Section | Status | Notes |
|---|---|---|
| **L1** Work model | mostly done | Work items with an active pointer (M1.1); issue intake (M2.1); queue, parking and blocked items (M2.2) on the new loop; no grace period yet |
| **L2** Agent loop | partial | `control.py`, a fixed state machine with no-stranding, `stop`, `abandon`. Replaced by the new loop (Phase 2) |
| **L3** Authority | partial | Revision binding and arrival stamping (`gates.py`); scope is still plan approval, not the issue and its conversation |
| **L2** Agent loop | mostly done | The loop (Phase 2), the only one since the switch-over (M2.8): driver runs, waits, `stop`, `abandon`, question runs |
| **L3** Authority | partial | Go-aheads bound to the restatement's revision, on the issue and its conversation (M2.1); no action classes yet (M3.1) |
| **L4** Classifier | none | |
| **L5** Conversation | partial | Matrix transport, mentions, lenient commands; on the new loop, a thread per item, named and ambiguous commands, plain-language go-aheads (M2.3); little explanation |
| **L6** Context | partial | Context fitting, per-role views and orchestrator facts (M1.2); no scope or agent memory |
@@ -41,7 +41,7 @@ Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
| **D5** Review | partial | PR conversation comments only (#69), watch stall (#70) |
| **D7–D8** Deployment, projects | partial | Runs from a venv as a systemd unit on this box; project entries in config (M1.4); egress not enforced |
| **O**, **X** | none | No container image yet |
| `specs/` | done for now | `orchestrator.qnt` (current loop), `loop.qnt` (the architecture, #91) |
| `specs/` | done for now | `loop.qnt` (the architecture, #91); `orchestrator.qnt` went with `control.py` (M2.8) |
---
@@ -51,8 +51,9 @@ Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
Authelia. The infrastructure repository exists in Gitea.
- **Local inference:** llama-swap on an RTX 4090 host (320 W cap). The benchmark
(PR #55) chose `qwen3.8-27b` for local work.
- **Models by role today:** executor `qwen3.8-27b`, planner and helper
`claude-opus-5-5`, expert `claude-haiku-4-5`. Phase 2 moves to driver and helpers.
- **Models by role today:** driver and helper `claude-opus-5-5`, expert
`claude-haiku-4-5`. `qwen3.8-27b` has had no role since the executor's went
(M2.8); the driver benchmark measures where it fits.
- **Bring-up** runs on this development VM (`scripts/bringup.sh`), against test
resources: a test repository, the test Matrix room, a budget-capped LiteLLM key;
configuration in `/root/nisse-e2e/`.
@@ -85,11 +86,9 @@ Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
(`-k`) or diagnosed (`--lf`); `scripts/e2e.sh` reports what a run spent. Manual
live runs are for the new loop's first runs and phase ends. Findings are filed as
issues and either fixed before moving on or deliberately deferred.
- **The Quint models move with the loop:** `orchestrator.qnt` while `control.py`
runs, `loop.qnt` for the new loop, whose properties become tests as each piece
lands (Phase 2).
- **`control.py` gets fixes only,** and only for what blocks a live run. New
behaviour goes into the new loop.
- **The Quint model moves with the loop:** `loop.qnt` models the architecture,
and its properties are tests (the property walk), changed in the same PR as a
rule they cover.
- **Bring-up runs on this box as root,** against test resources only, with today's
gates on until the classifier and trust dial are proven (Phase 3).
- **This plan is revised as we go,** in the same PR as the work that changes it.
@@ -230,8 +229,8 @@ deletes afterwards.
**M2.6 Roles and capability profiles.** *Pre-approved.* **Done** (this PR).
`driver`, `helper`, `expert`, `classifier` (L9); escalation triggers; delegation and
`ask_expert` on the new loop.
- `roles.driver` (unset: the planner's model) and `roles.classifier` (unset: the
expert's), pinned per item; the driver and the go-ahead reader run on the item's.
- `roles.driver` (required since M2.8; until then, unset meant the planner's
model) and `roles.classifier` (unset: the expert's), pinned per item; the driver and the go-ahead reader run on the item's.
- `profiles:` in configuration (tools, strengths, weaknesses), used in briefings:
the driver is told what its helper and expert are good at, the helper who will
read its handback; never a model's name.
@@ -264,9 +263,8 @@ in prose, asking mid-work (L5, #42, #77).
defaults, secrets by name with their visibility, egress additions.
- Acceptance: the bring-up configuration is one project entry; a second project
needs only a second entry.
- `projects: {name: entry}`. The old loop runs exactly one project and refuses
more. On the new loop each project has its own clone, workspace, push credential
and milestones. Issues are named `project#14` when two projects could both have
- `projects: {name: entry}`. Each project has its own clone, workspace, push
credential and milestones. Issues are named `project#14` when two projects could both have
`#14`, and a bare `#14` that matches two is asked about. Switching projects parks
the one worked on last.
- Secrets are by `${VAR}` reference only, `commands` or `harness`. `commands`
@@ -295,8 +293,20 @@ the release's own tests over it).
- The live bring-up's switch to the image waits for the next live test (it needs
its own test repository too); `bringup.sh plan` shows what it will run.
**M2.8 Switch over.** *Pre-approved, after acceptance.* `nisse-dev` runs the new
loop; `control.py`, its tests and `orchestrator.qnt` are removed.
**M2.8 Switch over.** *Pre-approved, after acceptance.* **Done** (this PR, after
the live session's fixes in #119 and #122). `nisse-dev` runs only the loop.
- Removed: `control.py`, `planner.py`, `executor.py`, `approvals.py`, `gates.py`,
`publish.py`, `poller.py`, the old assembly (`agent.py`, folded into
`loop/build.py`), the state store's state-machine view, their tests, the old
end-to-end and live planning tests, and `orchestrator.qnt` with its scenarios.
- Configuration: `loop:`, `roles.planner`, `roles.executor` and
`tuning.executor` are refused with what to use instead; `roles.driver` is
required, and `tuning.driver` takes the driver's turn cap and failure limit.
- The benchmark and the capability matrix measure the driver: the loop's own
system prompt and tools, the task's plan as the agreed restatement. Task plans
end with `finish_milestone`, so earlier runs show as `(old)`.
- A delegation on the loop records what its helper's alias resolved to (L9), as
the old path's did.
### Phase 3 — Authority
@@ -392,7 +402,7 @@ if the implementation differs.
lines, then drop the oldest events with a note saying so; never the system
prompt, the current plan or the most recent turns. View labels:
`[orchestrator]`, `[operator, Matrix]`, `[operator, comment on PR #N]`,
`[helper report]`, `[executor]` narration.
`[helper report]`, `[driver]` narration.
- **L7 rendering:** content is stored as bytes or local paths and uploaded or
inlined at render time, never as provider handles. Token budgeting and
prompt-cache breakpoints are render-time concerns, never stored. Compaction
+44 -36
View File
@@ -127,9 +127,10 @@ a local model can be held to less without affecting the frontier roles.
response inside the context window. With a 32k context and a 16k ceiling,
only 16k is left for the prompt. Some servers reject any request where
prompt plus ceiling exceeds the context.
- **Too low:** truncated plans and tool calls.
- **Deciding:** executor turns are short, so 2–4k is plenty for a local
executor. Frontier planning wants the large default. Leave this high, and
- **Too low:** truncated restatements and tool calls.
- **Deciding:** a working turn is short, so 2–4k is plenty for a local
model doing a milestone's work. A frontier driver restating and designing
wants the large default. Leave this high, and
set a local model's smaller ceiling as its `max_output_tokens` in LiteLLM.
### `llm.request_timeout_s` — default 600
@@ -137,8 +138,8 @@ a local model can be held to less without affecting the frontier roles.
How long one model request may take before it is abandoned. It applies to
every request, on both routes.
- **Too low:** a slow model's long answer is cut off, and the executor
escalates work that was going well. At 120 s, the old fixed value, a dense
- **Too low:** a slow model's long answer is cut off, and the driver's run
fails on work that was going well. At 120 s, the old fixed value, a dense
27B model on the RTX 4090 timed out writing one module in one turn.
- **Too high:** a hung connection holds the milestone for that long before
anything notices.
@@ -147,10 +148,12 @@ every request, on both routes.
expect, which is the model's output ceiling divided by its generation
speed.
### `tuning.executor.max_turns` — default 40
### `tuning.driver.max_turns` — default 40
How many model calls one milestone may make before the executor stops with
`capped` and asks you what to do.
How many model calls one driver run may make before it stops as `capped` and
asks you what to do. A run lasts until the driver waits (for you, a review or
a merge), so a long milestone is usually one run. Before M2.8 this was
`tuning.executor.max_turns`.
- **Too low:** milestones that were making progress stop short. You'll see
`capped` outcomes on work that was going well.
@@ -167,12 +170,14 @@ How many model calls one milestone may make before the executor stops with
`long-milestone`, one of which had in fact finished the work. It is a guard
against a model trying forever, not a budget, so it was raised to 40.
### `tuning.executor.failure_limit` — default 2
### `tuning.driver.failure_limit` — default 2
How many consecutive failed `run_command` calls trigger escalation: the
sub-task is handed to a frontier model as a delegation (architecture L9). A
successful command resets the count. Failed file reads don't count, since
reading is looking around, not doing the work.
How many consecutive failed `run_command` calls mean the driver is stuck: the
first time in a milestone, a helper takes the problem as a delegation
(architecture L9); the second time, the item waits on you. A successful
command resets the count. Failed file reads don't count, since reading is
looking around, not doing the work. Before M2.8 this was
`tuning.executor.failure_limit`.
- **Why it's a count and not a judgment:** small models are poorly calibrated
about their own competence. Asked "are you stuck?", they say no until far
@@ -194,13 +199,13 @@ How many `ask_expert` calls one milestone may make. `ask_expert` is a single
stateless question to a stronger model, with no access to the repository
(architecture L9), for things like "is this API deprecated?".
- **Too low:** the executor can't get unstuck on a knowledge question it
- **Too low:** the driver can't get unstuck on a knowledge question it
can't answer from the code.
- **Too high:** a confused executor asks variations of the same question
- **Too high:** a confused driver asks variations of the same question
instead of doing the work or failing. That's the circling L9 warns about.
- **Deciding:** count consults per milestone, and check whether the answers
changed what the executor did next. If it regularly exhausts the budget
without progress, the budget is fine and the model or the plan is the
changed what the driver did next. If it regularly exhausts the budget
without progress, the budget is fine and the model or the brief is the
problem.
### `tuning.delegation` — hard caps on one delegation
@@ -282,10 +287,9 @@ Tokens are estimated from characters (pessimistically, at 3 per token) and
the estimate is corrected from the prompt sizes the server reports. The
correction only ever makes the estimate larger.
The executor also sends less in the first place: the approved milestone plan
goes in its system prompt, which is never cut, and it sees only what happened
since that plan. The planning conversation and earlier milestones stay in the
log.
Each role also sends less in the first place: it sees its own work item's
transcript (L6), with other roles' work reduced to what it needs (a helper's
handback is one report to the driver).
| Knob | Default | |
|---|---|---|
@@ -331,14 +335,15 @@ Asking a model that can't honour it can be worse than not asking: the
Qwen3-14B turn above costs a full output ceiling of generation, where a plain
prose reply would have cost a few hundred tokens.
Where it isn't enforced, the executor holds the contract itself:
Where it isn't enforced, the driver's run holds the contract itself:
- **`finish` is the only way to end a milestone.** It is refused once if
nothing has been changed yet.
- **A prose reply is a violation.** The model is told once; a second escalates.
- **A call to a tool that wasn't offered counts as a failure** toward
`failure_limit`, like a failed command, so an inventing model escalates in a
couple of turns instead of using up `max_turns`.
- **A waiting tool is the only way to end a run** (`finish_milestone`,
`ask_human`, and the like); the harness, not the model, then publishes or
asks.
- **A prose reply is a violation.** The model is told once; a second ends
the run, and the item waits on you.
- **A call to a tool that wasn't offered is answered, not run:** the result
names the tools there are, so the model can put it right on its next turn.
The capability matrix checks the declaration: a model declared `true` that
answers a required call in prose fails
@@ -412,14 +417,17 @@ done.
The first four were the original set. By September 2026 all seven newer local
models passed all of them, so the last four were added to separate models that
can hold the tool contract from models that can do a milestone's real work.
Their plans stay explicit, as the planner is told to write them; what makes
Their plans stay explicit, as an agreed restatement is; what makes
them harder is the work itself: tracing a cause, searching a codebase,
reading a spec closely, and holding a plan over many turns.
It runs the executor exactly as the agent does (the same client wiring,
context budget and `tool_choice` handling, and your config's tuning knobs),
with **delegation and `ask_expert` turned off**. An escalation is a failure of
the model under test, not a frontier model's success.
It runs the loop's own driver exactly as the agent does: the same system
prompt, tools, client wiring, context budget, `tool_choice` handling and your
config's tuning knobs. The task's plan is given as the agreed restatement, and
finishing the milestone ends the run. **Delegation and `ask_expert` are
turned off**: an escalation is a failure of the model under test, not a
frontier model's success. Until M2.8 it ran the old loop's executor, so runs
from before then are against older task versions and show as `(old)`.
**Evaluating a new model** is one command, once its LiteLLM entry declares its
limits:
@@ -456,7 +464,7 @@ served differently (different note) are kept apart.
`failure_limit: 2` mid-recovery. That says as much about the knob as about
the model. Tuning a knob and re-running is a fair comparison, because the
tuning is recorded with every run.
- **Cost** is per run, straight from LiteLLM, so a frontier model as executor
- **Cost** is per run, straight from LiteLLM, so a frontier model as driver
can be compared on success rate *and* dollars.
---
@@ -511,8 +519,8 @@ Later work that will add knobs, each with its own section here:
- **Compaction.** A model-written summary replacing old turns, instead of
dropping them. The log already has an event for it; it needs a summarizing
model and a policy for what is safe to summarize.
- **Benchmarking the other roles:** planner and delegate, with delegation
turned back on for comparisons across frontier tiers.
- **Benchmarking the other roles:** the helper, with delegation turned back
on for comparisons across frontier tiers.
### `tuning.loop.bug_grace_s` — default 600