nisse-ui (X6) shows the operator each agent's work and, per item, a walkthrough down to every model request and response, for studying how context, compaction and memory work. It reads each agent's state store, call record and memory folder through read-only mounts, so agent code is unchanged and each agent stays the only writer of its state; it is released with the agents and tested against stores their own code wrote. The call record (L7) is new in nisse-core: every request as sent, the raw response with its reasoning, timing, tokens, prompt reuse and each message's provenance. It is diagnostic only, never read by the agent, and deleted at start, so it never migrates. Commands from the page, including a hold on a task or a whole agent, are an open question (Part Q). The shared database first proposed in #109 is dropped. The plan gets M6.6. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
907 lines
56 KiB
Markdown
907 lines
56 KiB
Markdown
# Nisse — Development Plan
|
||
|
||
**Status: agreed (PR #84); reworked in PR #92 after the architecture revision (PR
|
||
#90).** How to get from the code as it is to the architecture in
|
||
`docs/Architecture.md`, in what order, and how each step is accepted. The
|
||
architecture says *what and why*; this document says *how and when*, and is revised
|
||
as work lands.
|
||
|
||
**The priority is a minimal new loop, running live.** Work is ordered to reach the
|
||
smallest end-to-end version of the conversational loop (M2.1) as soon as
|
||
possible; everything not needed for it waits. Each milestone is checked by an
|
||
automated end-to-end test against real resources, not by hand.
|
||
|
||
---
|
||
|
||
## 1. Where we are
|
||
|
||
A survey of `main` (2026-10-02) against the architecture. **Done** means it exists
|
||
and matches; **partial**, something exists but needs changing; **none**, not
|
||
started.
|
||
|
||
Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
|
||
|
||
| Section | Status | Notes |
|
||
|---|---|---|
|
||
| **L1** Work model | mostly done | Work items with an active pointer (M1.1); issue intake (M2.1); queue, parking and blocked items (M2.2) on the new loop; no grace period yet |
|
||
| **L2** Agent loop | mostly done | The loop (Phase 2), the only one since the switch-over (M2.8): driver runs, waits, `stop`, `abandon`, question runs |
|
||
| **L3** Authority | partial | Go-aheads bound to the restatement's revision, on the issue and its conversation (M2.1); action classes and the fast path, with `allow` bound to one dangerous action (M3.1); the trust dial, `trust.merge` (M3.3) |
|
||
| **L4** Classifier | mostly done | On the classifier role (the local model), for what the fast path doesn't know; fails closed; overrides export as an evaluation set (M3.2). Promoting its verdicts into rules is by hand |
|
||
| **L5** Conversation | partial | Matrix transport, mentions, lenient commands; on the new loop, a thread per item, named and ambiguous commands, plain-language go-aheads (M2.3); little explanation |
|
||
| **L6** Context | partial | Context fitting, per-role views and orchestrator facts (M1.2); no scope or agent memory |
|
||
| **L7** Canonical log | partial | Work item and actor on every event, `fact` (M1.1, M1.2); `action_verdict` (M3.1); no `knowledge_retrieved` |
|
||
| **L8** Durability | mostly done | WAL store, intents, boot reconciliation, PR adoption, comment redelivery; to move onto the new loop |
|
||
| **L9** Models and roles | mostly done | LiteLLM, eval capture; on the new loop: `driver`/`helper`/`expert`/`classifier` pinned per item, capability profiles in briefings, delegation and `ask_expert` with triggers (M2.6) |
|
||
| **L10** Knowledge vault | none | |
|
||
| **L11** Injection | partial | Operator allowlists on every channel; no classifier |
|
||
| **L12** Secrets, logging | partial | Structured state-change log; secrets in `.env`; no per-operation handout or redaction |
|
||
| **L13** Execution | none | Tools run locally in the agent's process. The token leaks into the clone (#89) |
|
||
| **D1–D2** Workflow | partial | On the new loop: issue intake, restatement and go-ahead (M2.1), labels (M2.3); no documents in the repo |
|
||
| **D3–D4** Workspace, git | partial | Idempotent publishing; no fresh workspace (#62), `git add -A` (#80) |
|
||
| **D5** Review | partial | PR conversation comments only (#69), watch stall (#70) |
|
||
| **D7–D8** Deployment, projects | partial | Bring-up runs `nisse-dev`'s and `nisse-review`'s images under systemd on this box; project entries in config (M1.4); egress not enforced |
|
||
| **O**, **X** | partial | `nisse-dev` has its image (M1.5), `nisse-review` its own (M7.6, acceptance after merge); nothing runs on the agent host |
|
||
| `specs/` | done for now | `loop.qnt` (the architecture, #91); `orchestrator.qnt` went with `control.py` (M2.8) |
|
||
|
||
---
|
||
|
||
## 2. The environment today
|
||
|
||
- **Supporting services** (X5): Matrix, Gitea, LiteLLM, Vaultwarden, Samba AD
|
||
with Authelia. The infrastructure repository exists in Gitea.
|
||
- **Local inference:** llama-swap on an RTX 4090 host (320 W cap). The benchmark
|
||
(PR #55) chose `qwen3.8-27b` for local work.
|
||
- **Models by role today:** driver and helper `claude-opus-5-5`, expert
|
||
`claude-haiku-5-5`, classifier `qwen3.8-27b` (M3.2); the driver benchmark
|
||
measures where the local model fits beyond that.
|
||
- **Bring-up** runs on this development VM (`scripts/bringup.sh`), against test
|
||
resources: a test repository, the test Matrix room, a budget-capped LiteLLM key;
|
||
configuration in `/root/nisse-e2e/`.
|
||
- **Test repositories are made per run** by the test operator
|
||
(`nisse-testing-operator`, `NISSE_OPERATOR_TOKEN`), as an operator provides a
|
||
project's: seeded from `tests/integration/seed/`, the agent a collaborator,
|
||
`main` protected, a deploy key of its own; deleted afterwards
|
||
(`scripts/e2e-cleanup.sh` for leftovers). They replaced the shared sandbox
|
||
repository, whose state accumulated across runs.
|
||
- **A build VM for bring-up** is provided by the operator by hand until Terraform
|
||
(Phase 6); the agent reaches it over SSH with a key from `/root/nisse-e2e/ssh/`.
|
||
|
||
---
|
||
|
||
## 3. How we work
|
||
|
||
- **One PR per milestone,** each starting with a short implementation plan per
|
||
`CLAUDE.md`. A milestone marked **needs discussion** starts with a design
|
||
discussion first.
|
||
- **Every change comes with tests,** through `scripts/check.sh`.
|
||
- **Testing has three levels** (M1.6): every PR runs the checks (unit tests across
|
||
the runner's cores, 16 property walks from a moving seed, integration) and the
|
||
end-to-end **smoke** tests; a PR labelled `e2e:full`, a push to `main` and a
|
||
manual run add all 48 walks and the whole end-to-end suite; a published release
|
||
runs that full set before its image is built. The end-to-end tests drive the
|
||
real agent, every role on a local model (so they cost nothing per call), in a
|
||
repository of the run's own, with a scripted operator; CI runs them after the
|
||
checks pass. There is one runner, so a newer run for a
|
||
PR or branch cancels the one it supersedes (#126). Locally only the tests being written
|
||
(`-k`) or diagnosed (`--lf`); `scripts/e2e.sh` reports what a run spent. Manual
|
||
live runs are for the new loop's first runs and phase ends. Findings are filed as
|
||
issues and either fixed before moving on or deliberately deferred.
|
||
- **Every agent is its own container image** (X1), since that's how each one
|
||
runs on the agent host: built from the repository's `Dockerfile`, published by
|
||
`release.yml` to `gitea.dongar.net/dev/<agent>`, and tested over the image the
|
||
way `nisse-dev`'s is (M1.5). A new agent's phase includes its image, before the
|
||
agent host is ready for it, and its bring-up runs from that image.
|
||
- **The Quint model moves with the loop:** `loop.qnt` models the architecture,
|
||
and its properties are tests (the property walk), changed in the same PR as a
|
||
rule they cover.
|
||
- **Bring-up runs on this box as root,** against test resources only, with today's
|
||
gates on until the classifier and trust dial are proven (Phase 3).
|
||
- **This plan is revised as we go,** in the same PR as the work that changes it.
|
||
|
||
---
|
||
|
||
## 4. Phases
|
||
|
||
### Phase 0 — Housekeeping (done)
|
||
|
||
**M0.1 Retire the old spec.** **Done.** **M0.2 A bring-up script.** **Done.**
|
||
|
||
### Phase 1 — Execution over SSH, and an end-to-end test
|
||
|
||
**M1.1 Work items in the state store.** **Done.**
|
||
|
||
**M1.2 Transcript views.** **Done.** (#78 confirmed live, and closed.)
|
||
|
||
**M1.3 Execution over SSH.** *Pre-approved.* **Done** (#93).
|
||
- `execute(host, command, *, secrets, stdin)` as the only way to run anything; file
|
||
tools over SSH; the host bound by the harness from the project (L13).
|
||
- The workspace lives on the build VM: cloned there, with no token in its
|
||
configuration; the push key handed to each push only (D4, #89).
|
||
- Every secret handed out is redacted from tool output (L12).
|
||
- Tests against a throwaway `sshd` container.
|
||
- Acceptance: the existing executor and delegation tests pass over SSH; a live run
|
||
of one milestone on the operator's build VM.
|
||
- Also landed, from that run (bring-up run 3): the push credential as an HTTPS
|
||
token or an SSH deploy key; flat harness branch names, after the executor made
|
||
a branch named after the task; **publishing only the work (#80)**, brought
|
||
forward from M2.1; credentials kept out of every repr.
|
||
|
||
**M1.6 The end-to-end test.** *Pre-approved.* **Done** (#95): `scripts/e2e.sh`.
|
||
- Automates bring-up run 3: the assembled agent, real models, the Gitea sandbox and
|
||
a real build host, with a scripted operator that answers whatever the agent asks.
|
||
- Asserts what was checked by hand: a PR on the sandbox containing only the work,
|
||
pushed with the push credential; no credential in the clone; the planner's next
|
||
turn citing the facts rather than the executor's turns.
|
||
- Runs against the current loop now, and against the new loop from M2.1.
|
||
|
||
**Phase 1 acceptance:** the end-to-end test passes against the current loop.
|
||
**Done:** it passes on the bring-up build VM.
|
||
|
||
**Moved after M2.1:** M1.4 (projects) and M1.5 (the container image). Neither is
|
||
needed for a minimal loop running live; both are done before the switch-over
|
||
(M2.8), when several projects and deployment start to matter.
|
||
|
||
### Phase 2 — The new loop, as a thin slice first
|
||
|
||
Built alongside `control.py`, reusing the state store, log, views, publisher,
|
||
pollers and recovery. The first milestone is the smallest end-to-end path; each
|
||
later one widens it and extends the end-to-end test (M1.6) to cover it.
|
||
|
||
**M2.1 The slice.** *Needs discussion* (the loop's shape in code). **Done** (#96,
|
||
#97, #98, #99; first live run: sandbox #255, follow-ups in #100).
|
||
- An issue assigned to `dev-agent` → a work item (D1); a restatement in the item's
|
||
thread; a go-ahead bound to its revision (L3, L5); one milestone through
|
||
`start_milestone` / `finish_milestone` on a harness branch (D4); the PR; a review
|
||
comment answered (D5); the merge; the item closed.
|
||
- Includes what carries over: a fresh workspace at item start and harness-owned
|
||
branches (D3, D4, #62); review intake, inline comments and the watch stall
|
||
(#69, #70); the review-fix check after merge. (Publishing only the work, #80,
|
||
landed early in M1.3.)
|
||
- `loop.qnt`'s properties for these paths become tests.
|
||
- Acceptance: the end-to-end test runs that path on the new loop, plus a crash
|
||
mid-publish; then a first manual live run.
|
||
|
||
**M2.2 The queue.** *Pre-approved.* **Done** (this PR). Several items: states,
|
||
park-and-switch with branches pushed, next-item order, blocked items from Gitea
|
||
dependencies, a closed PR handled (L1, D4). End-to-end: two issues and a dependency.
|
||
- Switching happens as soon as the active item waits; the grace period (§6, L1)
|
||
is left until a watching operator misses it. Order is "do #N next"; priority
|
||
labels and driver-chosen order wait for a need.
|
||
- The property walk covers several issues, dependencies and the operator's order,
|
||
and checks that parked work is never lost (every file an open item wrote is on
|
||
the workspace or in the forge).
|
||
|
||
**M2.3 Threads and labels.** *Pre-approved.* **Done** (this PR). Routing by thread,
|
||
ambiguous commands asked about (L5); `bug` grace periods, `discuss`, `design`;
|
||
questions and plain-language go-aheads (D1). End-to-end: a bug and a feature,
|
||
labelled and not.
|
||
- Commands naming their item ("abandon #14") apply to it anywhere; `go`, `approve`
|
||
and `abandon` alone at top level are asked about. `stop` is never an answer.
|
||
- A plain-language go-ahead is read by one model call (`GoAheadReader`, on the
|
||
driver's model until the classifier role, M2.6); questions never reach it.
|
||
- `bug` waits `tuning.loop.bug_grace_s` (default 10 minutes) after its restatement;
|
||
any reply stops the countdown. `discuss` isn't offered `restate` until the
|
||
operator has spoken; `design` is steered by the driver's instructions.
|
||
- Labels are read at intake; a label changed later is not picked up yet.
|
||
|
||
**M2.4 Multi-milestone work and documents.** *Pre-approved.* **Done** (this PR).
|
||
Build plans in the repository, `ARCHITECTURE.md` kept current, waiting for merges
|
||
between milestones (D2, D4). End-to-end: a two-milestone feature.
|
||
- Every milestone branch is cut from the latest default branch, so the next one
|
||
builds on the merged previous and a redo drops a closed PR's work. While the
|
||
item's PR is open, `start_milestone` is refused, the PR's state read from Gitea
|
||
at that moment. `finish_milestone(final)`: earlier PRs say "Part of #N", the
|
||
last "Closes #N".
|
||
- `close_item` with the item's PR still open waits for the merge.
|
||
- Documents are the driver's instructions only; the harness never parses a plan.
|
||
- Parking stashes uncommitted work and pushes it to `<item>-parked`, put back on
|
||
resume, instead of committing it onto the item's branch: the walk found a
|
||
half-done review fix parked onto a PR that had merged meanwhile, where it was
|
||
lost.
|
||
- Deferred: a milestone with no changes that wasn't expected to change code is
|
||
still raised with the operator (D4).
|
||
|
||
**M2.5 Conversations and new projects.** **Done** (#106 and this PR). *Needs discussion;* the design was agreed
|
||
in this PR (`general` scope, L1, L5; the new-project sequence, D1). Work started in
|
||
conversation, record issues carrying their item's ID, the new-project flow and its
|
||
project entry (D1, D8, #29). After M1.4. End-to-end: a small new project, in a
|
||
repository the test operator creates under its own account for the run and
|
||
deletes afterwards.
|
||
- **PR A, conversations — done (this PR).**
|
||
- A top-level message that isn't a command is a `general` item, threaded under
|
||
it; the driver gets `reply`, `ask_human` and `propose_work`, with no workspace.
|
||
- Agreed work (a go-ahead, or a plain yes, on the proposal) is filed under an
|
||
intent as a record issue carrying the new item's ID. Its item starts agreed,
|
||
with no second restatement. Intake skips any issue carrying an ID, and boot
|
||
settles a filing cut off by a crash.
|
||
- A conversation closes after `tuning.loop.general_idle_s` without a message; a
|
||
reply in its thread then starts a new one that continues it.
|
||
- **PR B, new projects — done (this PR).**
|
||
- `propose_project(name, purpose, decisions)` goes up for a go-ahead.
|
||
- On go, the checklist: repository, protection, collaborator, build VM, deploy
|
||
key, and the project entry as YAML to fill in.
|
||
- On "done", the agent reads its configuration again and adds the new project
|
||
live (only additions). It then checks the access: the repository's
|
||
permissions and protection as the agent sees them, then in practice on a
|
||
throwaway branch with an empty commit (a branch push must succeed, a push to
|
||
the default branch and merging its own PR must be refused); the branch and PR
|
||
are removed. Anything that fails is said, and nothing starts.
|
||
- When all is well: the tracking issue with the distilled decisions, and the
|
||
project's item, agreed and labelled `design`, whose first milestone is
|
||
`ARCHITECTURE.md` and the build plan. A conversation waiting on provisioning
|
||
doesn't close as idle.
|
||
|
||
**M2.6 Roles and capability profiles.** *Pre-approved.* **Done** (this PR).
|
||
`driver`, `helper`, `expert`, `classifier` (L9); escalation triggers; delegation and
|
||
`ask_expert` on the new loop.
|
||
- `roles.driver` (required since M2.8; until then, unset meant the planner's
|
||
model) and `roles.classifier` (unset: the expert's), pinned per item; the driver and the go-ahead reader run on the item's.
|
||
- `profiles:` in configuration (tools, strengths, weaknesses), used in briefings:
|
||
the driver is told what its helper and expert are good at, the helper who will
|
||
read its handback; never a model's name.
|
||
- `delegate(task)` and `ask_expert(question)` for the driver. A helper works on a
|
||
branch off the milestone's; its work comes back as uncommitted changes on a
|
||
handback and is discarded otherwise, so harness branches never hold commits only
|
||
the clone has (the walk found a fix stranded that way when a PR merged mid-help).
|
||
- Triggers: `failure_limit` failed commands in a row, or the turn cap, mid-
|
||
milestone, call a helper in once per milestone; if it fails too, the item waits
|
||
on the operator as before. A delegation cut off by a crash is discarded at boot.
|
||
- Evaluation capture: delegations, `ask_expert` and the reader's verdicts.
|
||
- Deferred: the context-won't-fit trigger (L6); the action classifier (M3.2).
|
||
|
||
**M2.7 Conversation.** *Pre-approved.* **Done** (this PR). Explanations, progress
|
||
in prose, asking mid-work (L5, #42, #77).
|
||
- Questions never change state: a message ending in `?` to a waiting project item
|
||
is answered in a question run (`reply` and look-only tools), and the item goes
|
||
back to the very wait it was in, so a restatement up stays up and a go binds to
|
||
it.
|
||
- A restatement gives its reasons and the main alternative; a revision says what
|
||
changed (`restate(changed)`, sent back without it).
|
||
- Progress: the driver's and a helper's own words for a turn are posted, trimmed,
|
||
at most once a minute per item; `details` shows the last tool calls. A message
|
||
arriving mid-run is acknowledged at once.
|
||
- Deferred: streaming (#42); the question property in `specs/loop.qnt` (the walk
|
||
checks it against the code).
|
||
|
||
**M1.4 Projects** (D8). *Pre-approved.* **Done** (this PR).
|
||
- The project entry in configuration: repository, default branch, build VM, label
|
||
defaults, secrets by name with their visibility, egress additions.
|
||
- Acceptance: the bring-up configuration is one project entry; a second project
|
||
needs only a second entry.
|
||
- `projects: {name: entry}`. Each project has its own clone, workspace, push
|
||
credential and milestones. Issues are named `project#14` when two projects could both have
|
||
`#14`, and a bare `#14` that matches two is asked about. Switching projects parks
|
||
the one worked on last.
|
||
- Secrets are by `${VAR}` reference only, `commands` or `harness`. `commands`
|
||
secrets reach `run_command` as environment variables, redacted from output;
|
||
`harness` secrets are held for later. Egress is recorded, not enforced (D7).
|
||
- Deferred: config reload without a restart (with M2.5's entry step if wanted);
|
||
the trust dial's configuration (M3.3).
|
||
|
||
**M1.5 The agent as a container image.** *Pre-approved.* **Done** (#115; accepted
|
||
2026-10-07: `v0.1.0`'s published image passed the whole e2e suite, 9 of 9, run with
|
||
the release's own tests over it).
|
||
- A Dockerfile for `nisse-dev`, stateless with a volume for state; a Gitea Actions
|
||
workflow that builds it and pushes to Gitea's registry on each release tag (X4).
|
||
- `bringup.sh` runs the image instead of a venv.
|
||
- Acceptance: a tagged release produces an image; the end-to-end test runs from it.
|
||
- The image runs as its own user; state in `/var/lib/nisse-dev`, configuration
|
||
mounted at `/etc/nisse-dev/nisse-dev.yaml`, secrets as environment variables,
|
||
keys mounted read-only. Its `test` stage carries the tests over the installed
|
||
package, for `scripts/e2e.sh --image REF`. Images are built only on release:
|
||
`release.yml` pushes `gitea.dongar.net/dev/nisse-dev:<version>` and `:latest`
|
||
on a published release (`REGISTRY_TOKEN`, the `dev` organization's).
|
||
- Acceptance runs the tests over a published image: `docker build --target test
|
||
--build-arg TEST_BASE=gitea.dongar.net/dev/nisse-dev:<version>` from the
|
||
release's checkout, then `scripts/e2e.sh --image`; the test stage adds only the
|
||
test tools, so the agent under test is the one published.
|
||
- The live bring-up's switch to the image waits for the next live test (it needs
|
||
its own test repository too); `bringup.sh plan` shows what it will run.
|
||
|
||
**M2.8 Switch over.** *Pre-approved, after acceptance.* **Done** (this PR, after
|
||
the live session's fixes in #119 and #122). `nisse-dev` runs only the loop.
|
||
- Removed: `control.py`, `planner.py`, `executor.py`, `approvals.py`, `gates.py`,
|
||
`publish.py`, `poller.py`, the old assembly (`agent.py`, folded into
|
||
`loop/build.py`), the state store's state-machine view, their tests, the old
|
||
end-to-end and live planning tests, and `orchestrator.qnt` with its scenarios.
|
||
- Configuration: `loop:`, `roles.planner`, `roles.executor` and
|
||
`tuning.executor` are refused with what to use instead; `roles.driver` is
|
||
required, and `tuning.driver` takes the driver's turn cap and failure limit.
|
||
- The benchmark and the capability matrix measure the driver: the loop's own
|
||
system prompt and tools, the task's plan as the agreed restatement. Task plans
|
||
end with `finish_milestone`, so earlier runs show as `(old)`.
|
||
- A delegation on the loop records what its helper's alias resolved to (L9), as
|
||
the old path's did.
|
||
|
||
### Phase 3 — Authority
|
||
|
||
**M3.1 Action classes and the fast path.** *Needs discussion.* **Done** (this PR).
|
||
Harmless, contained and dangerous actions per agent (P3, D6), static fast-path rules.
|
||
- `nisse_core.authority.actions` (the shared part, `nisse-core`'s first code): `Action`,
|
||
`Verdict`, and reading shell commands statically, segment by segment (`cd`,
|
||
wrappers like `sudo` and `timeout`, `sh -c`); what can't be read that way is
|
||
unknown, and a line is as bad as its worst command.
|
||
- `nisse_dev.loop.actions`, the rule table: reading, checks and tests harmless;
|
||
workspace edits, local git and installs on the build VM contained; pushing,
|
||
other hosts, uploads, publishing, the forge's tools, deleting outside the
|
||
workspace and wrecking the VM dangerous; changing branches by hand refused.
|
||
- Every execution call is screened before it runs, the driver's and the
|
||
helper's. A dangerous one isn't run: the item waits on the operator with a
|
||
question rendered from the action, and `allow` lets exactly that action run
|
||
once. A helper can't ask, so it is refused and told to hand back.
|
||
- Every verdict is an `action_verdict` event (no model sees it) and an oplog
|
||
line; `details` shows the counts and every action not simply run.
|
||
- Decided: unknown actions run, logged, until the classifier (M3.2), since this
|
||
is a project in progress, not production.
|
||
- Deferred: the classifier and in-scope judgement (M3.2), the trust dial
|
||
(M3.3), egress enforcement (X2).
|
||
|
||
**M3.2 The classifier.** *Needs discussion.* **Done** (this PR). L4, its model,
|
||
fail-closed verdicts, overrides as an evaluation set.
|
||
- `nisse_dev.loop.classifier`: one model call per action the fast path calls
|
||
unknown, on the item's pinned `classifier` role. It sees the action and the
|
||
agreed scope (project, repository, issue number, the restatement the operator
|
||
said go to), built in code: never the transcript, the issue's text, tool
|
||
output or the driver's reasoning. Its verdict is a tool call: `safe` runs,
|
||
`in_scope` runs and is reported in the thread, `needs_discussion` asks as a
|
||
dangerous action does. An error, no verdict or an unknown one fails closed.
|
||
- Decided: no different-model requirement (separate context is what L4's
|
||
independence needs; the verdicts record the model, so correlated misses can
|
||
be measured); the local model to start with; verdicts cached per item,
|
||
agreed revision and action.
|
||
- `action_verdict` gains `source` (fast path or classifier), `model` and the
|
||
classifier's exact `input`; `details` shows the source. An `allow` after the
|
||
classifier asked is an override. `scripts/classifier-evals.py` exports the
|
||
judgements from a store (read-only); `tests/integration/classifier_cases.yaml`
|
||
is the seed set, run per model with `--capability-models`
|
||
(`qwen3.8-27b-nothink` and `claude-haiku-4-5`: 9 of 9 each).
|
||
- The fast path's delete rule: on the build VM, deleting is contained; only
|
||
what breaks the host stays dangerous (decided with the operator).
|
||
- Deferred: promoting verdicts the classifier keeps agreeing on into rules (by
|
||
hand, from exported cases); the trust dial (M3.3).
|
||
|
||
**M3.3 The trust dial.** *Needs discussion.* **Done** (this PR). Per project (D8);
|
||
agent merging behind protection it can't satisfy, verified by a test (D4, D5, #65).
|
||
- `trust.merge` per project: `operator` (the default) or `approved`: the harness
|
||
(never the model) merges a PR in the PR watch once an approving review from a
|
||
`gitea.operators` account is on the PR's current commit, its checks pass (no
|
||
checks at all counts as passing; pending waits) and it's mergeable; then the
|
||
item goes on as after any merge. Each reason to hold off is said once.
|
||
- Protection, verified at start per `approved` project: a throwaway branch and
|
||
PR made through the API (never the workspace), whose merge must be refused;
|
||
if it isn't, the operator merges until a restart. Real Gitea is tested too
|
||
(`test_trust_live.py`): refused unapproved, merged once approved, refused
|
||
again after a commit lands on the approved PR.
|
||
- Test and bring-up repositories get the protection: approvals from an
|
||
operators-only whitelist, stale approvals dismissed, the agent on the merge
|
||
whitelist.
|
||
- Decided: two levels only (no merging on checks alone, D5); verification by
|
||
behaviour, since a collaborator can't read protection and must never set it.
|
||
Deferred: a project-setup script for the operator (protection, labels) and a
|
||
label check, #130.
|
||
|
||
### Phase 4 — The environment
|
||
|
||
**M4.1 Toolchain preflight, M4.2 the toolchain step, M4.3 environment repair,
|
||
M4.4 an unreachable build VM.**
|
||
*Pre-approved.* All on the build VM (D3).
|
||
|
||
**M4.1** is **done** (this PR). A model works out and sets up what a project
|
||
needs; code makes sure that happens, bounds it and checks the result.
|
||
- Before each driver run on a project, one round trip checks the build VM: the
|
||
harness's own baseline (missing: the item waits, a template problem), the
|
||
workspace clone (missing: cloned again, said, and the item resumes from what
|
||
it pushed) and a per-project marker (`~/.nisse/<project>.ready`).
|
||
- No marker (a new project, a new or wiped VM): the item's driver model runs
|
||
the preflight first, with its own prompt (the project's toolchain, not the
|
||
template's basics or tools that act on other systems; never credentials),
|
||
the execution tools (screened as ever) and `toolchain_ready`. The harness
|
||
runs each reported version command itself (version-shaped commands only,
|
||
wherever they come from) and writes the marker. Gaps and an unfinished
|
||
preflight are facts, and work goes on; the thread hears only of installs and
|
||
gaps. A rebuilt VM gets the last toolchain as its starting point.
|
||
- Later runs check the versions again; a change is a fact and a thread line,
|
||
and the PR body says what the work was tested with.
|
||
- A changed host key is its own failure: the item waits for the operator's
|
||
`trust new host key`, then the VM is set up again (since M4.4, `vm ready`).
|
||
- Decided with the operator: the model judges toolchains (the harness can't
|
||
without it), preflight runs on the driver's model, gaps don't block. Turning
|
||
off unattended upgrades and the template's baseline are template choices.
|
||
|
||
**M4.2** *(revised with the operator: in place of recording installs).*
|
||
**Done** (this PR). The
|
||
preflight becomes **the toolchain step** (D3): make the VM and the project's own
|
||
files agree, with the repository as the record rather than a harness-kept one.
|
||
- Started by the harness when the VM isn't set up for the project (M4.1's
|
||
checks): it sets the VM up from the repository and edits nothing there.
|
||
- Requested by the driver (`toolchain_step(reason)`) once work is agreed: it
|
||
installs, then brings pins, setup docs and CI configuration in step; the edits
|
||
land in the current milestone's PR. Refused before the go-ahead.
|
||
- Run in a transcript of its own; the driver sees only its report.
|
||
- With it, D1's rule made structural: until the go-ahead the fast path refuses
|
||
anything that isn't harmless, so needs go in the restatement.
|
||
|
||
**M4.3** *(revised with the operator: a broken VM blocks its projects).*
|
||
**Done** (this PR). Cleaning caches and reinstalling dependencies were already
|
||
contained and reported as the actions they are, so what is left is a VM broken
|
||
past the usual fixes (D3).
|
||
- The driver can `request_reimage(reason)`, before the go-ahead too (it asks,
|
||
it doesn't act). Its prompt says to try the usual fixes first, then ask rather
|
||
than go on diagnosing.
|
||
- The harness parks the work first (pushed to Gitea, as for a switch; a VM too
|
||
broken to push loses it, and the operator is told), then **blocks the build
|
||
VM**: a durable block per host, so every project on it waits and none of its
|
||
items is taken up, across restarts. Said once at top level with the projects
|
||
blocked and the command that clears it, and in each blocked item's thread.
|
||
- The operator's `vm ready <project>` (no name needed when only one is blocked)
|
||
lifts the block and trusts the VM's new host key. A reimaged VM has no marker,
|
||
so the toolchain step sets it up again before work; one said ready without a
|
||
reimage goes straight on. The item that found it goes first.
|
||
- `loop.qnt` gained the block (`blocked_vm_runs_nothing`); it found that work in
|
||
progress was lost on a reimage, hence parking first. The walk requests
|
||
reimages and recovers them, reimaged or not.
|
||
- Later, nisse-ops reimages a VM itself (M6.2, M8.5); a changed address means
|
||
editing `build_host` and restarting.
|
||
|
||
**M4.4** *(#137, with the operator).* **Done** (this PR). A VM that can't be
|
||
reached blocks through M4.3's mechanism; only the triggers and the way out
|
||
differ.
|
||
- Triggers: the check before a run, a run that loses the VM (it ends there, so
|
||
no model works against a missing VM: the toolchain step and a requested step
|
||
too), and preparing the workspace. The item stays ready, waiting on no one.
|
||
- Tried again every `tuning.build_vm.retry_s` (5 min) on the intake round, said
|
||
only in the operations log; once it answers, it must still answer
|
||
`settle_s` (3 min) later, so a VM that has just booted is left to finish.
|
||
Then the work a lost run left on it is parked (pushed), before any item can
|
||
set it aside, and it is unblocked and said once. `vm ready` clears it too;
|
||
the message warns that a reimage loses what isn't pushed.
|
||
- A changed host key, at the check or on a retry, blocks the VM as well, and is
|
||
cleared only by `vm ready`, which trusts the new key: `trust new host key` is
|
||
retired, and the message says it's a key problem and that it may be an
|
||
impersonation.
|
||
- Questions are answered while a VM is blocked, without it (L5: a question
|
||
changes nothing). Starting the agent with a VM down no longer fails.
|
||
- The walk breaks the VM's SSH, now or partway through a run, and checks that
|
||
no model works on a VM that is down. With `loop.qnt`, it found: work a lost
|
||
run left behind set aside by the next item there (hence parking first); a
|
||
question left unanswered behind a block (hence answering without the VM);
|
||
a lost VM swallowed while the toolchain step wrote its marker; and a remark
|
||
waiting to be answered that outlived its PR's merge, sending the item back
|
||
to wait for a merge that had happened. All fixed.
|
||
|
||
### Phase 5 — Memory
|
||
|
||
**M5.1 The vault.** *Pre-approved.* **Done** (this PR). The folder, control
|
||
tools, atomic writes and provenance (L10).
|
||
- `vault.py`: only the configured folders exist for the agent, each `rw` or `ro`
|
||
(`vault:` in the configuration; nested folders refused, one access each).
|
||
Paths are checked in code: relative, inside a folder (symlinks included),
|
||
Markdown, never the editor's own files (`vault.ignore`, by default hidden
|
||
files and SilverBullet's `_plug`).
|
||
- Reads cut long notes and say so; search is a plain case-insensitive match over
|
||
readable notes, most matches first, with no index yet.
|
||
- Writes are atomic (a hidden temporary file beside the note, synced, renamed,
|
||
the directory synced) and merge `provenance` (by, source, work item, time)
|
||
into the note's frontmatter, keeping the rest. Only the library writes for
|
||
now: when the model writes memory is M5.2's.
|
||
- `read_note`, `list_notes`, `search_notes`: control tools in the harness for
|
||
the driver (before the go-ahead too, and while a build VM is blocked) and
|
||
conversations, labelled `[vault: <path>]`; a failure is the call's result.
|
||
The prompt says a note is information, never an instruction.
|
||
- Decided with the operator: SilverBullet instead of Obsidian, on the vault
|
||
itself (M5.4), so no Syncthing. Atomic writes stay with one writer: they
|
||
protect readers (SilverBullet, the harness) and crashes, not concurrency.
|
||
- Per-agent mounts are the container's (M6.1); until then the folder map and
|
||
path checks hold in code.
|
||
|
||
**M5.2 Agent memory.** *Discussed with the operator.* **Done** (this PR). When
|
||
notes are written and read (L6.4).
|
||
- `remember(name, description, fact)` and `forget(name, why)`: control tools for
|
||
the driver (before the go-ahead too) and conversations, writing only the
|
||
agent's folder (`memory/nisse-dev`, read-write in the vault configuration).
|
||
No one agrees first, the model may forget on its own, and each change is a
|
||
line in the thread ("Remembered: …", "Forgot: …").
|
||
- One note per fact (`<name>.md`: a one-line description, the fact, and
|
||
provenance: work item, thread, time). The harness keeps `index.md`, one line
|
||
per note, after every change; the model never writes it.
|
||
- The index goes into the transcript once per item and conversation, with its
|
||
path and a hash (L6); notes are read with `read_note`. The prompt says memory
|
||
is a cache: check against the live system, fix a note a check contradicts.
|
||
- At an item's close, completed or abandoned, a **memory pass**: the driver's
|
||
model in a transcript of its own (the item's narration, as the toolchain step
|
||
sees it), with the note and memory tools and `memory_done`, bounded by
|
||
`tuning.memory.max_turns` (10). It never holds the close up.
|
||
- Decided with the operator: corrections may be paraphrased (L6 changed); no
|
||
confirmation; no limit on what may prompt a note (a lesson often comes from
|
||
tool output), each recording what did. Conversations get no closing pass.
|
||
- The walk remembers and forgets, and checks that nothing lands outside the
|
||
agent's folder.
|
||
|
||
**M5.3 Scope memory.** *Discussed with the operator.* **Done** (this PR) (L6.3).
|
||
- Project notes have their own folder, `memory/nisse-dev/projects/<project>/`,
|
||
with its own index; `remember` and `forget` take `scope` (`general`, or
|
||
`project`: always the current item's, so no item writes another project's
|
||
notes; a conversation has none). At an item's start the transcript gets the
|
||
general index and its project's, never another project's.
|
||
- Index links are pages from the vault's root, as SilverBullet names them.
|
||
- The closing pass's `memory_done` can name repository documents the item left
|
||
out of date: said in the thread and recorded, for the operator.
|
||
- The prompt: the repository is the authority; a note disagreeing with it is
|
||
corrected; don't remember what it already holds.
|
||
- Decided with the operator: `ARCHITECTURE.md` isn't put into every item's
|
||
context (its size); the driver still reads it as told. The infrastructure
|
||
world model waits for `nisse-ops` (Phase 8).
|
||
- The walk writes project notes and checks they come only from that project's
|
||
items.
|
||
|
||
**M5.4 The operator's view.** *Pre-approved.* SilverBullet in its own container on
|
||
the vault: the operator's folders read-write, the agents' read-only (L10). Done
|
||
early on the agent host (`deploy/agent-host`), so the operator can keep material
|
||
for `nisse-ops` there now:
|
||
- The vault is `/srv/nisse/vault`. The operator owns its root and everything in
|
||
it but `memory/`, the agents'; `infrastructure/` is the operator's material
|
||
for `nisse-ops`. SilverBullet serves the vault as its one space (single-space
|
||
mode, mounted at `/data`), with `memory/` mounted over it read-only, and sees
|
||
nothing else of the host.
|
||
- SilverBullet's version is pinned (the docker role updates images nightly).
|
||
It has no accounts and no setup wizard; `SB_NAME` forces single-space mode,
|
||
so an empty vault can't boot the wizard.
|
||
- Decided with the operator: behind their Caddy at `nisse-memory.dongar.net` for
|
||
HTTPS, reachable from anywhere through Authelia, which is the login (agents
|
||
never go through SilverBullet, so this changes only the operator's access).
|
||
Its own instance rather than the operator's personal one, so the vault stays
|
||
one local copy with mounts enforcing its writers. On the host, a
|
||
`DOCKER-USER` rule lets only Caddy reach it, so nothing gets around Authelia.
|
||
|
||
### Phase 6 — Production
|
||
|
||
*Infrastructure (2026-10-18):* VMs are now made by PR to `infra/tofu`
|
||
(OpenTofu), and the agent host exists (M6.1), holding only the vault and
|
||
SilverBullet so far (M5.4). Until M6.2, the build VMs stay made by hand, with
|
||
`vm ready` for a rebuilt one's host key.
|
||
|
||
**M6.1 The agent host.** *Needs discussion.* Containers, volumes, snapshots,
|
||
per-agent networks (X1, X2, X4). Each agent runs there from its own published
|
||
image (§3). The host exists: `nisse` (infra/tofu#77, 2026-10-18, as
|
||
`nisse-host`; 2 cores, 4 GB, 100 GB, on VLAN 1, nightly Proxmox backups),
|
||
requested through nisse-dev's own quota to test that path; the operator has
|
||
since renamed it and moved it out of the quota. It holds only the vault and
|
||
SilverBullet so far (M5.4).
|
||
|
||
**M6.2 Build VMs by Terraform.** *Needs discussion.* The template, VLAN and egress
|
||
(D7, X2), collection when idle.
|
||
|
||
**M6.3 Secrets and identity.** *Pre-approved.* Per-agent secret sets on the host,
|
||
Samba AD principal (L12).
|
||
|
||
**M6.4 Delivery.** *Pre-approved.* Watchtower, restarts gated on an idle flag (X4).
|
||
|
||
**M6.5 The browser tool.** *Pre-approved.* On the internal test network (D7).
|
||
|
||
**M6.6 The status page.** *Pre-approved (design agreed 2026-10-24, #109).*
|
||
`nisse-ui` (X6), for studying how the agents work: context, compaction, memory.
|
||
Slices a–c are built on this box before the agent host is ready, against
|
||
bring-up, end-to-end and benchmark runs; only d waits for M6.1.
|
||
- **a. The call record** (L7) in `nisse-core`: written by the shared model client
|
||
and `Driver.run`, with each message's provenance from the per-role view and
|
||
fitting; wired in each agent's build; deleted at start.
|
||
- **b. The walkthrough:** an item's steps, each run's turns, and each request read
|
||
against the log it was made from.
|
||
- **c. The overview:** each agent's active, queued, parked and blocked items.
|
||
- **d. Deployment:** its image, the read-only mounts and a Caddy route behind
|
||
Authelia (with #166).
|
||
|
||
Commands wait (Part Q). The shared database #109 first proposed is dropped: each
|
||
agent keeps its own store.
|
||
|
||
### Phase 7 — `nisse-core` and `nisse-review`
|
||
|
||
`nisse-review` (Part R) comes before `nisse-ops`: it writes only comments, so it
|
||
needs none of ops' machinery or infrastructure, while being a real second user of
|
||
the loop. Decided with the operator, 2026-10-13.
|
||
|
||
**M7.1 Extract `nisse-core`.** *Done.* What isn't the dev agent's moved to
|
||
`nisse-core`, unchanged in behaviour, with its tests: the transcript and its
|
||
renderers, the LLM client and model catalog, context fitting and views, the state
|
||
store, reconciliation, the worker and `stop`, transports (Matrix), SSH execution,
|
||
the vault and memory, logging, `.env` loading, and the Gitea client (whole: L1
|
||
makes Gitea every agent's tracker). `nisse-core` never imports an agent (a test
|
||
says so).
|
||
|
||
- `nisse_core.state.store.StateStore` holds the tables every agent has; an agent's
|
||
subclass adds its own tables (with their schema version and migrations), its
|
||
model roles, and what each role is shown of the others (`views`).
|
||
`nisse_dev.state.StateStore` adds the build VM blocks (D3); stores already at
|
||
version 5 open unchanged.
|
||
- The vault names whose writes it records (`agent`), rather than assuming one.
|
||
- Deferred until a second agent has a loop to share: splitting the loop
|
||
(`loop/*`) and configuration into a common part and the dev agent's. The dev
|
||
agent's `git`, delegation, `ask_expert`, executor tools and evaluation capture
|
||
stay in `nisse-dev`.
|
||
|
||
**Core packages.** *Done.* `nisse-core`'s modules grouped into packages by
|
||
architecture section (`models`, `log`, `state`, `knowledge`, `conversation`,
|
||
`execution`, `authority`), as a pure move ahead of M7.2.
|
||
|
||
**M7.2 Share the loop.** *Done.* Shared parts, not a shared scheduler: each
|
||
agent keeps its own (nisse-review's is small and different), and a common one
|
||
waits until two exist to compare. Moved, unchanged in behaviour:
|
||
|
||
- `nisse_core.loop.driver`: the driver run. The agent names its waiting tools
|
||
per run, and execution goes to any `Executor` (nisse-dev's `Workspace` is
|
||
one).
|
||
- `nisse_core.loop.tools`: the control-tool helper and the tools any agent
|
||
offers (notes, memory, `memory_done`, `ask_expert`); `nisse_core.loop.expert`.
|
||
- `nisse_core.knowledge.handlers`: `Knowledge`, the vault and memory as
|
||
control tools, the memory index and the prompt text; the agent frames the
|
||
memory pass.
|
||
- `nisse_core.config`: reading the file, references, people, tuning sections
|
||
of the agent's own, the LLM's numbers, the vault.
|
||
- Stays in `nisse-dev` (decided with the operator): the scheduler, items,
|
||
milestones, preflight, the go-ahead reader, its fast-path rules, the
|
||
classifier (until `nisse-ops`), delegation, profiles and projects.
|
||
|
||
**M7.3 `nisse-review`'s first slice.** *Done.* The `nisse-review` package, over
|
||
`nisse-core`:
|
||
|
||
- **Duty and items (R1):** each round, the open PRs in every repository its
|
||
account can read (or the configured `repositories`); a work item per PR,
|
||
ready unless a draft or titled WIP; closed when the PR merges or closes.
|
||
Every open PR is reviewed, those open when it starts included (operator).
|
||
Since #153, `repositories` holds patterns (`dev/*`, `*/*`) matched against
|
||
every repository it can read, public ones included, through Gitea's search
|
||
(`/repos/search`, which needs no `read:organization` scope); the read-only wall
|
||
is checked every round, not only at startup.
|
||
- **Reading (R3):** the repository at the PR's head and base, from Gitea's
|
||
archives, unpacked into a scratch folder and read in Python (`read_file`,
|
||
`list_files`, `search_code`; a long file is read in parts, `start` giving
|
||
the line, since a file cut at 20,000 characters left its end out of reach),
|
||
and its issues (`read_issue`,
|
||
`list_issues`): no shell, bound to the item's repository. Since the M7.5
|
||
follow-ups, the code comes from a git clone kept between runs, not an
|
||
archive per run (see there).
|
||
- **A review:** starts from the PR, its commits, diff, comments, the issues it
|
||
closes and CI; `comment` drafts a point on any line of any file (checked to
|
||
exist), `post_review` posts one COMMENT review at the head. Gitea takes
|
||
comments on the diff inline; one it refuses off the diff goes into the
|
||
summary as a link to that line (operator: comment on any line).
|
||
`ask_expert`, and the vault and memory (`memory/nisse-review`, notes per
|
||
repository).
|
||
- **Durability (L8):** posting is an intent; after a crash, Gitea is asked
|
||
whether the review (marked with its head) was posted, and nothing is posted
|
||
twice; with Gitea down, the item waits.
|
||
- **Failures:** a failed run is tried again; after `tuning.review.attempts`, it
|
||
says so on the PR and waits for the next push. Each run starts afresh from its
|
||
brief (the log keeps the rest), and one out of turns with points drafted posts
|
||
them as a partial review: on #156 the reviewer ran out of turns, then its retry,
|
||
carrying every read before it, timed out, and both drafts were lost.
|
||
- **Access:** startup refuses an account that can push to, or administer, any
|
||
repository it reads. The token needs `read:user`, `write:issue` and
|
||
`write:repository` scopes; the account's read access is the limit.
|
||
- **Bring-up:** `scripts/review-bringup.sh`, a transient systemd unit from the
|
||
exported code; config and secrets in `/root/nisse-review`.
|
||
- **Tests:** unit tests over a fake Gitea and a scripted model; a live test
|
||
(`test_review_live.py`, `NISSE_REVIEW_GITEA_TOKEN`, the test reviewer
|
||
account's) has the local model review a PR in a fresh test repository. It is a
|
||
smoke test, run by `scripts/e2e.sh` on every PR (until #152 it never ran in CI:
|
||
the script ran only the loop's file).
|
||
|
||
**M7.4 `nisse-dev` answers a peer.** *Done.* Anyone's comment but an operator's
|
||
is a peer's point (D5, L11), not only `nisse-review`'s: settled with the operator,
|
||
since only accounts they made can log in to Gitea, with the model deciding whether
|
||
a bot's comment needs a reply.
|
||
- Each top-level comment, review body and inline comment is a point (P1, P2, …)
|
||
on the item, in the transcript as a `peer` message labelled with its author
|
||
(`nisse_core.log.views`). Its own comments are skipped.
|
||
- `answer_peer` answers where the point was made: inline on its line, or quoting
|
||
it. `skip_peer` records why no reply is needed. The thread gets one line per
|
||
run on what was answered and skipped.
|
||
- A run going back to waiting with points open is reminded once; then the harness
|
||
lists them on the PR. A merge or close drops the rest, with a fact.
|
||
- A peer's comment wakes the item; a wake by peers alone returns to the same wait
|
||
without a note to the operator. The operator's inline comments are now answered
|
||
inline too.
|
||
- **Tests:** `test_loop_peers.py` over the fake Gitea.
|
||
|
||
**M7.5 Discussion and rounds.** *Done.* Later reviews of what changed and
|
||
of earlier points, replies on threads, the round cap and its reset, `stop` and
|
||
`review` on a PR (R1, R2, R5):
|
||
|
||
- **What wakes it:** each round, intake reads what was said on every open PR
|
||
(top-level comments, review bodies, line comments) and keeps the keys it has
|
||
taken in; anything new but its own is a point to answer. A run is due for a
|
||
PR never reviewed (or `review`ed), a new head (a **later review**, after its
|
||
CI as before), or new points at the same head (**replies**, no CI wait).
|
||
- **A later review** is given the commits since its last review with each
|
||
one's diff (Gitea's compare lists them; the per-commit diff comes through the
|
||
API, since the web's compare diff takes no token), or the whole diff if they
|
||
can't be listed, and its earlier points with what was said on their lines.
|
||
- **Replies:** `reply` answers a point where it was made (inline on its line,
|
||
else quoting it), `skip` leaves it with a reason, `done` ends; every point is
|
||
answered or skipped before a run may end. An ending call refused for that is
|
||
held, and goes ahead as soon as the last point is settled, rather than left
|
||
to the model to make again (on #166 it didn't, and looped to the turn cap).
|
||
Each reply carries its point's key, so a run re-done after a crash answers
|
||
nothing twice.
|
||
- **Settled threads:** each `reply` says whether it settles its thread (a
|
||
concession, or an answer or deferral accepted; never while waiting on a
|
||
change). A settled reply carries `<!-- nisse-review: settled -->`. Being
|
||
read-only (R3), the reviewer can't resolve the thread, so the PR's author
|
||
does: nisse-dev resolves every comment on that thread's line when it next
|
||
watches the PR and takes the settled reply as no point to answer (nor one
|
||
posted top-level, which has no thread to resolve);
|
||
`scripts/resolve_settled.py` does the same for PRs made by hand.
|
||
- **Sign-off:** `sign_off` posts a short review when there is nothing to raise;
|
||
while it stays happy, a later review offers `still_happy`, which posts nothing
|
||
(operator: no repeated LGTMs).
|
||
- **Rounds:** each review, and each run that replied, is a round. Past
|
||
`tuning.review.max_rounds`, the next run sums up what is still open for the
|
||
operator (`post_summary`, as a review) and the item goes quiet (`capped`). A
|
||
push, or a comment from anyone but the reviewer and the PR's author (bots
|
||
included: the operator wants to see how the model handles them), resets it.
|
||
- **`stop` and `review`:** an operator's whole comment; acknowledged with a 👍
|
||
reaction. `stop` quiets the item through pushes and comments; `review` asks
|
||
for a fresh review of the whole PR, and retries a failed item.
|
||
- **Durability:** reviews carry a numbered marker (`<!-- nisse-review: {head}
|
||
#{n} -->`, the first in M7.3's form), so recovery asks Gitea about the right
|
||
one; items from before M7.5 take what was already said as heard.
|
||
- **Tests:** `test_review_rounds.py`, including a no-dead-end walk over every
|
||
position; the live smoke test adds a push and a question, and checks the
|
||
later review at the new head.
|
||
- **Follow-ups:** a stop (SIGTERM) ends a review in progress at its next turn,
|
||
or breaks off its model call (`Interruptible`, `nisse_core.models`), and the
|
||
PR goes back to ready with no failure counted; a repository Gitea won't
|
||
list is logged when that starts or changes (`intake.repo_failed`), and when
|
||
it ends (`intake.repo_ok`), not every round. Tests: `test_review_stop.py`.
|
||
- **Code from a kept clone** (operator: archives don't scale to very large
|
||
repositories): each repository is cloned once, with its whole history, under
|
||
`scratch/repos/<owner>/<name>/` (`head/` the clone, `base/` a worktree made
|
||
when read). A run fetches every branch and the PR's `refs/pull/N/head`,
|
||
checks out the head exactly (`checkout --force`, `clean -ffdx`), and fails if
|
||
that head is gone. Only the latest repository is kept for now; the layout
|
||
holds several. Git runs walled off: none of the host's git configuration,
|
||
links as plain files, no hooks, submodules or LFS, only the Gitea URL's
|
||
protocol, and the token as an `http.extraHeader` for that URL in git's
|
||
environment only. A stop ends a git command, and a cut-short first clone is
|
||
removed. `review.clone` / `review.fetch` log the time and size. No size
|
||
limits yet. Tests: `test_clones.py`, against local bare repositories.
|
||
- **Compaction** (#158; reviews of PRs touching about 20 files hit the turn cap,
|
||
re-reading files fitting had cut): shared by both agents' driver runs, as in
|
||
L6 above. Each cut fitting still makes is logged (`context.trimmed`).
|
||
Tests: `test_compact.py` in nisse-core.
|
||
|
||
**M7.6 `nisse-review`'s image.** *Pre-approved.* Doesn't wait for the agent
|
||
host. Its own image, as `nisse-dev`'s (M1.5, §3): a `nisse-review` target in the
|
||
`Dockerfile`, running as its own user, with state in a volume, configuration
|
||
mounted read-only and secrets as environment variables. The image needs `git`,
|
||
and `scratch/` a persistent volume, so clones outlive a restart. `release.yml`
|
||
publishes `gitea.dongar.net/dev/nisse-review` beside `nisse-dev`, and
|
||
`review-bringup.sh` runs the image instead of a virtualenv.
|
||
- Acceptance: a release publishes both images, and the review end-to-end tests
|
||
run over the published `nisse-review` image.
|
||
- The `review` target: user `nisse-review` (uid 10002), state and its clones
|
||
(`scratch/`) in `/var/lib/nisse-review`, configuration at
|
||
`/etc/nisse-review/nisse-review.yaml`, only its own agent installed. One
|
||
release builds every agent's image at one version: they share `nisse-core`
|
||
and were tested together. Older versions go by the `dev` organization's
|
||
package cleanup rule (keep the 10 most recent, and `latest`), set in Gitea.
|
||
- The `test` stage adds an agent package its base lacks without its
|
||
dependencies, since the suite's own code imports every agent's: over
|
||
`nisse-review`'s image, `scripts/e2e.sh --image REF -k test_review_live`.
|
||
`tests/deploy/test_images.py` (needs Docker) checks that what the base
|
||
installed stays as it was.
|
||
- The bring-up runs the image as root with `$REVIEW` at its own path, as
|
||
`bringup.sh` runs `nisse-dev`, so its config and state stay where they were.
|
||
|
||
**M7.7 Deployment.** *Waits for the agent host (Phase 6).* Its container on the
|
||
agent host, Gitea account and secrets (R5).
|
||
|
||
### Phase 8 — `nisse-ops`
|
||
|
||
**M8.1 MCP servers** (#37). Settled so far: redaction scrubs each instance's own
|
||
credentials (L12), with masking what an API returns as #144; search-then-dispatch
|
||
meta-tools are avoided, not ruled out (X3). **M8.2 Approved step sequences** (O2).
|
||
**M8.3 Ops intake and duties** from the infrastructure repository (O5), taking over
|
||
the Terraform the operator is writing now. **M8.4 The ops tool VM and deployment**
|
||
(O6); its own image comes with its first slice, as every agent's does (§3).
|
||
**M8.5 Requests between agents:** `nisse-dev` asking for build VMs (L1, O5). All
|
||
*needs discussion*.
|
||
|
||
### Not yet scheduled
|
||
|
||
From the architecture's open questions and older issues: filing issues from
|
||
conversation; stacked PRs; releases; `nisse-ops` issuing credentials; a server
|
||
database; recovering the agent host (an ops issue); more than one active work item;
|
||
commands from the status page, with a hold on a task or agent (X6);
|
||
vision; one local model for every agent; several forges (#26); a room per project
|
||
(#28).
|
||
|
||
---
|
||
|
||
## 5. Open issues and where they land
|
||
|
||
| Issue | Lands in |
|
||
|---|---|
|
||
| #26 multiple forges | Not yet scheduled |
|
||
| #27 per-project state, clone on use | M1.3 (clone on the build VM), M1.4 (projects) |
|
||
| #28 room per project | Not yet scheduled (threads, M2.3, may make it unnecessary) |
|
||
| #29 create a project from scratch | M2.5 (ownership settled by D4, D8) |
|
||
| #37 MCP servers | M8.1 |
|
||
| #42 per-turn commentary | M2.7 |
|
||
| #62 branch at execution start | M2.1 |
|
||
| #65 agent merges approved PRs | M3.3 |
|
||
| #69 inline review comments | M2.1 |
|
||
| #70 watch stall | M2.1 |
|
||
| #77 conversational agent | M2.3 (questions), M2.7 |
|
||
| #109 a UI for every agent's status (no shared database) | M6.6 |
|
||
| #137 block on an unreachable build VM | M4.4 |
|
||
| #144 redact secrets an MCP server returns | After M8.1 |
|
||
|
||
## 6. Implementation notes
|
||
|
||
Mechanism detail moved out of `docs/Architecture.md` in PR #90, which keeps only
|
||
decisions and their reasons. Keyed by the architecture section each note
|
||
implements; a milestone touching that section uses the note, and changes it here
|
||
if the implementation differs.
|
||
|
||
- **L1 switching:** the grace period before switching away from an item waiting
|
||
on the operator defaults to 10 minutes. When the driver chooses the next ready
|
||
item and the choice fails or is invalid, fall back to the oldest.
|
||
- **L3 revision stamping:** in Matrix, a reply is stamped with the revision whose
|
||
message had been sent when it arrived; on a Gitea issue, with the latest
|
||
proposal comment older than the reply.
|
||
- **L5 transport:** the `Transport` interface is send, ask and receive. Intake
|
||
backoff is exponential, capped at a few minutes.
|
||
- **L6 compaction (#158):** before each driver call, over `compact_at` (0.6) of
|
||
the prompt budget, the same model is asked, over the same request plus one
|
||
message, for notes on everything between what started the work and the last
|
||
turns (at most `compact_keep_turns`, 3, in `compact_to`, 0.25). The notes,
|
||
with code's list of the calls they replace, become a `compaction` event
|
||
that renders in place of its range; a later one folds in the earlier.
|
||
Failed notes are logged (`context.compaction_failed`) and fitting takes
|
||
over; `context.compacted` logs each one. Not in nisse-dev's helper loop,
|
||
on purpose (operator, 2026-10-18): its token cap (`delegation.max_tokens`,
|
||
200k, every call's whole prompt counted) ends a delegation at a prompt of
|
||
about 36k tokens, well short of `compact_at` on any window in use. Wire
|
||
`Compactor` in if that cap is raised several-fold.
|
||
- **L6 fitting order:** first shorten old tool results to their first and last
|
||
lines, then drop the oldest events with a note saying so; never the system
|
||
prompt, the current plan or the most recent turns. View labels:
|
||
`[orchestrator]`, `[operator, Matrix]`, `[operator, comment on PR #N]`,
|
||
`[helper report]`, `[driver]` narration.
|
||
- **L7 rendering:** content is stored as bytes or local paths and uploaded or
|
||
inlined at render time, never as provider handles. Token budgeting and
|
||
prompt-cache breakpoints are render-time concerns, never stored. Compaction
|
||
summaries are first-class events so a model change never inherits opaquely
|
||
compacted history. A reasoning "session" is a delegation, including one resumed
|
||
after a crash.
|
||
- **L8 state store:** SQLite in WAL mode, each change committed and fsynced before
|
||
the loop continues. A delegation's cut is clean when its last event is a
|
||
completed result, its branch is at the expected commit, its caps are unbreached
|
||
and external truth matches the log. "Anything else" offers continue, re-plan or
|
||
abandon.
|
||
- **L9 local models:** request `tool_choice` only of models whose server declares
|
||
it can enforce it; otherwise the loop enforces the contract (an explicit
|
||
`finish`, prose treated as a violation). Briefings use the capability profile,
|
||
never a model name or "weaker model".
|
||
- **L10 vault:** atomic writes are write-then-rename. SilverBullet's own files
|
||
(and `.git`, if history is kept) are never notes to an agent. A failed write is
|
||
the tool's result, never the run's end.
|
||
- **L12:** secrets reach a command by environment or stdin. Headless agents get an
|
||
Authelia single-factor exception scoped by source address, group and target.
|
||
Logs are shipped to the host journal, not kept in containers.
|
||
- **D3 preflight:** a project's needs are read from its files (`Cargo.toml`,
|
||
`pyproject.toml`, `package.json`) or a declaration in the repository.
|
||
- **D4 publishing:** standard excludes are configured in each clone, and only what
|
||
the work produced is committed. The push credential is handed to each push
|
||
only: a deploy key over stdin to a short-lived `ssh-agent` on the build VM, or
|
||
a push-only account's HTTPS token as that command's secrets, read by a
|
||
credential helper that names environment variables, never values.
|
||
- **D7:** test services use ports 18000–18099, so opening access later is a
|
||
firewall change. The inference host is behind LiteLLM, so it is not a second
|
||
egress exception.
|
||
- **X4:** the agent host's state dataset is ZFS, snapshotted on a schedule and on
|
||
each approval. Tool VM templates use Terraform's `bpg/proxmox` provider with
|
||
cloud-init. Containers restart unless stopped.
|
||
- **X5:** the local inference host is llama-swap on an RTX 4090 machine.
|