nisse/docs/Development-Plan.md

# Nisse — Development Plan

**Status: agreed (PR #84); reworked in PR #92 after the architecture revision (PR #90).** How
to get from the code as it is to the architecture in `docs/Architecture.md`, in
what order, and how each step is accepted. The architecture says *what and why*;
this document says *how and when*, and is revised as work lands.

**The priority is live testing.** Work is ordered so the agent runs live against
real resources as early and as often as possible, and what each run finds steers
what comes next.

---

## 1. Where we are

A survey of `main` (2026-10-02) against the architecture. **Done** means it exists
and matches; **partial**, something exists but needs changing; **none**, not
started.

Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.

| Section | Status | Notes |
|---|---|---|
| **L1** Work model | partial | Work items table with an active pointer (M1.1); no issue intake, queue, parking or blocked items |
| **L2** Agent loop | partial | `control.py`, a fixed state machine with no-stranding, `stop`, `abandon`. Replaced by the new loop (Phase 2) |
| **L3** Authority | partial | Revision binding and arrival stamping (`gates.py`); scope is still plan approval, not the issue and its conversation |
| **L4** Classifier | none | |
| **L5** Conversation | partial | Matrix transport, mentions, lenient commands; no threads, no question handling, little explanation |
| **L6** Context | partial | Context fitting, per-role views and orchestrator facts (M1.2); no scope or agent memory |
| **L7** Canonical log | partial | Work item and actor on every event, `fact` (M1.1, M1.2); no `classifier_verdict`, `knowledge_retrieved` |
| **L8** Durability | mostly done | WAL store, intents, boot reconciliation, PR adoption, comment redelivery; to move onto the new loop |
| **L9** Models and roles | partial | LiteLLM, delegation, `ask_expert`, eval capture; roles are still planner/executor, no capability profiles |
| **L10** Knowledge vault | none | |
| **L11** Injection | partial | Operator allowlists on every channel; no classifier |
| **L12** Secrets, logging | partial | Structured state-change log; secrets in `.env`; no per-operation handout or redaction |
| **L13** Execution | none | Tools run locally in the agent's process. The token leaks into the clone (#89) |
| **D1–D2** Workflow | none | Plans posted to a Gitea issue and gated; no issue intake, labels, or documents in the repo |
| **D3–D4** Workspace, git | partial | Idempotent publishing; no fresh workspace (#62), `git add -A` (#80) |
| **D5** Review | partial | PR conversation comments only (#69), watch stall (#70) |
| **D7–D8** Deployment, projects | none | Runs from a venv as a systemd unit on this box; one project in config |
| **O**, **X** | none | No container image yet |
| `specs/` | done for now | `orchestrator.qnt` (current loop), `loop.qnt` (the architecture, #91) |

---

## 2. The environment today

- **Supporting services** (X5): Matrix, Gitea, LiteLLM, Vaultwarden, LLDAP with
  Authelia. The infrastructure repository exists in Gitea.
- **Local inference:** llama-swap on an RTX 4090 host (320 W cap). The benchmark
  (PR #55) chose `qwen3.8-27b` for local work.
- **Models by role today:** executor `qwen3.8-27b`, planner and helper
  `claude-opus-5-5`, expert `claude-haiku-4-5`. Phase 2 moves to driver and helpers.
- **Bring-up** runs on this development VM (`scripts/bringup.sh`), against test
  resources: the sandbox repository, the test Matrix room, a budget-capped LiteLLM
  key; configuration in `/root/nisse-e2e/`.
- **A build VM for bring-up** is provided by the operator by hand until Terraform
  (Phase 6); the agent reaches it over SSH with a key from `/root/nisse-e2e/ssh/`.

---

## 3. How we work

- **One PR per milestone,** each starting with a short implementation plan per
  `CLAUDE.md`. A milestone marked **needs discussion** starts with a design
  discussion first.
- **Every change comes with tests,** through `scripts/check.sh`.
- **A live run follows each milestone that changes behaviour,** not just each phase:
  a short bring-up run of what the milestone added. Findings are filed as issues and
  either fixed before moving on or deliberately deferred.
- **The Quint models move with the loop:** `orchestrator.qnt` while `control.py`
  runs, `loop.qnt` for the new loop, whose properties become tests as each piece
  lands (Phase 2).
- **`control.py` gets fixes only,** and only for what blocks a live run. New
  behaviour goes into the new loop.
- **Bring-up runs on this box as root,** against test resources only, with today's
  gates on until the classifier and trust dial are proven (Phase 3).
- **This plan is revised as we go,** in the same PR as the work that changes it.

---

## 4. Phases

### Phase 0 — Housekeeping (done)

**M0.1 Retire the old spec.** **Done.** **M0.2 A bring-up script.** **Done.**

### Phase 1 — Execution over SSH, and the agent in a container

**M1.1 Work items in the state store.** **Done.**

**M1.2 Transcript views.** **Done.** (#78 closes when a live run confirms it.)

**M1.3 Execution over SSH.** *Pre-approved.*
- `execute(host, command, *, secrets, stdin)` as the only way to run anything; file
  tools over SSH; the host bound by the harness from the project (L13).
- The workspace lives on the build VM: cloned there, with no token in its
  configuration; the push key handed to each push only (D4, #89).
- Every secret handed out is redacted from tool output (L12).
- Tests against a throwaway `sshd` container.
- Acceptance: the existing executor and delegation tests pass over SSH; a live run
  of one milestone on the operator's build VM.

**M1.4 Projects** (D8). *Pre-approved.*
- The project entry in configuration: repository, default branch, build VM, label
  defaults, secrets by name with their visibility, egress additions.
- Acceptance: the bring-up configuration is one project entry; a second project
  needs only a second entry.

**M1.5 The agent as a container image.** *Pre-approved.*
- A Dockerfile for `nisse-dev`, stateless with a volume for state; a Gitea Actions
  workflow that builds it and pushes to Gitea's registry on each release tag (X4).
- `bringup.sh` runs the image instead of a venv.
- Acceptance: a tagged release produces an image; a live run from the image.

**Phase 1 acceptance:** a live run, from the image, of a milestone and a review
comment on the build VM.

### Phase 2 — The new loop, as a thin slice first

Built alongside `control.py`, reusing the state store, log, views, publisher,
pollers and recovery. The first milestone is the smallest end-to-end path; each
later one widens it and is followed by a live run.

**M2.1 The slice.** *Needs discussion* (the loop's shape in code).
- An issue assigned to `dev-agent` → a work item (D1); a restatement in the item's
  thread; a go-ahead bound to its revision (L3, L5); one milestone through
  `start_milestone` / `finish_milestone` on a harness branch (D4); the PR; a review
  comment answered (D5); the merge; the item closed.
- Includes what carries over: a fresh workspace at item start and harness-owned
  branches (D3, D4, #62); publishing only the work (#80); review intake, inline
  comments and the watch stall (#69, #70); the review-fix check after merge.
- `loop.qnt`'s properties for these paths become tests.
- Acceptance: a live run of that path, plus a crash mid-publish.

**M2.2 The queue.** *Pre-approved.* Several items: states, park-and-switch with
branches pushed, next-item order, blocked items from Gitea dependencies, a closed
PR handled (L1, D4). Live run with two issues and a dependency.

**M2.3 Threads and labels.** *Pre-approved.* Routing by thread, ambiguous commands
asked about (L5); `bug` grace periods, `discuss`, `design`; questions and
plain-language go-aheads (D1). Live run.

**M2.4 Multi-milestone work and documents.** *Pre-approved.* Build plans in the
repository, `ARCHITECTURE.md` kept current, waiting for merges between milestones
(D2, D4). Live run of a two-milestone feature.

**M2.5 Conversations and new projects.** *Needs discussion.* Work started in
conversation, record issues carrying their item's ID, the new-project flow and its
project entry (D1, D8, #29). Live run of a small new project.

**M2.6 Roles and capability profiles.** *Pre-approved.* `driver`, `helper`,
`expert`, `classifier` (L9); escalation triggers; delegation and `ask_expert` on the
new loop.

**M2.7 Conversation.** *Pre-approved.* Explanations, progress in prose, asking
mid-work (L5, #42, #77).

**M2.8 Switch over.** *Pre-approved, after acceptance.* `nisse-dev` runs the new
loop; `control.py`, its tests and `orchestrator.qnt` are removed.

### Phase 3 — Authority

**M3.1 Action classes and the fast path.** *Needs discussion.* Harmless, contained
and dangerous actions per agent (P3, D6), static fast-path rules.

**M3.2 The classifier.** *Needs discussion.* L4, its model, fail-closed verdicts,
overrides as an evaluation set.

**M3.3 The trust dial.** *Needs discussion.* Per project (D8); agent merging behind
protection it can't satisfy, verified by a test (D4, D5, #65).

### Phase 4 — The environment

**M4.1 Toolchain preflight, M4.2 recording installs, M4.3 environment repair.**
*Pre-approved.* All on the build VM (D3).

### Phase 5 — Memory

**M5.1 The vault.** *Pre-approved.* The folder, per-agent mounts, control tools,
atomic writes and provenance (L10).

**M5.2 Agent memory** (*needs discussion*) and **M5.3 scope memory**
(*pre-approved*): when notes are written and read, verbatim corrections (L6).

**M5.4 The operator's copy.** *Pre-approved.* Syncthing, directions per folder.

### Phase 6 — Production

**M6.1 The agent host.** *Needs discussion.* Containers, volumes, snapshots,
per-agent networks (X1, X2, X4).

**M6.2 Build VMs by Terraform.** *Needs discussion.* The template, VLAN and egress
(D7, X2), collection when idle.

**M6.3 Secrets and identity.** *Pre-approved.* Per-agent secret sets on the host,
LLDAP principal (L12).

**M6.4 Delivery.** *Pre-approved.* Watchtower, restarts gated on an idle flag (X4).

**M6.5 The browser tool.** *Pre-approved.* On the internal test network (D7).

### Phase 7 — `nisse-core` and `nisse-ops`

**M7.1 Extract `nisse-core`.** **M7.2 MCP servers** (#37). **M7.3 Approved step
sequences** (O2). **M7.4 Ops intake and duties** from the infrastructure
repository (O5). **M7.5 The ops tool VM and deployment** (O6). **M7.6 Requests
between agents:** `nisse-dev` asking for build VMs (L1, O5). All *needs discussion*
except M7.1.

### Not yet scheduled

From the architecture's open questions and older issues: filing issues from
conversation; stacked PRs; releases; `nisse-ops` issuing credentials; a server
database; recovering the agent host (an ops issue); more than one active work item;
vision; one local model for every agent; several forges (#26); a room per project
(#28).

---

## 5. Open issues and where they land

| Issue | Lands in |
|---|---|
| #26 multiple forges | Not yet scheduled |
| #27 per-project state, clone on use | M1.3 (clone on the build VM), M1.4 (projects) |
| #28 room per project | Not yet scheduled (threads, M2.3, may make it unnecessary) |
| #29 create a project from scratch | M2.5 (ownership settled by D4, D8) |
| #37 MCP servers | M7.2 |
| #42 per-turn commentary | M2.7 |
| #62 branch at execution start | M2.1 |
| #65 agent merges approved PRs | M3.3 |
| #69 inline review comments | M2.1 |
| #70 watch stall | M2.1 |
| #77 conversational agent | M2.3 (questions), M2.7 |
| #78 transcript scoping and attribution | Done in M1.1, M1.2; closes after a live run |
| #80 build artifacts in PRs | M2.1 |
| #89 token in the clone's `.git/config` | M1.3 |

## 6. Implementation notes

Mechanism detail moved out of `docs/Architecture.md` in PR #90, which keeps only
decisions and their reasons. Keyed by the architecture section each note
implements; a milestone touching that section uses the note, and changes it here
if the implementation differs.

- **L1 switching:** the grace period before switching away from an item waiting
  on the operator defaults to 10 minutes. When the driver chooses the next ready
  item and the choice fails or is invalid, fall back to the oldest.
- **L3 revision stamping:** in Matrix, a reply is stamped with the revision whose
  message had been sent when it arrived; on a Gitea issue, with the latest
  proposal comment older than the reply.
- **L5 transport:** the `Transport` interface is send, ask and receive. Intake
  backoff is exponential, capped at a few minutes.
- **L6 fitting order:** first shorten old tool results to their first and last
  lines, then drop the oldest events with a note saying so; never the system
  prompt, the current plan or the most recent turns. View labels:
  `[orchestrator]`, `[operator, Matrix]`, `[operator, comment on PR #N]`,
  `[helper report]`, `[executor]` narration.
- **L7 rendering:** content is stored as bytes or local paths and uploaded or
  inlined at render time, never as provider handles. Token budgeting and
  prompt-cache breakpoints are render-time concerns, never stored. Compaction
  summaries are first-class events so a model change never inherits opaquely
  compacted history. A reasoning "session" is a delegation, including one resumed
  after a crash.
- **L8 state store:** SQLite in WAL mode, each change committed and fsynced before
  the loop continues. A delegation's cut is clean when its last event is a
  completed result, its branch is at the expected commit, its caps are unbreached
  and external truth matches the log. "Anything else" offers continue, re-plan or
  abandon.
- **L9 local models:** request `tool_choice` only of models whose server declares
  it can enforce it; otherwise the loop enforces the contract (an explicit
  `finish`, prose treated as a violation). Briefings use the capability profile,
  never a model name or "weaker model".
- **L10 vault:** atomic writes are write-then-rename. Obsidian's settings folder
  (and `.git`, if history is kept) are not synced. A failed write is retried, or
  recorded and reconciled later.
- **L12:** secrets reach a command by environment or stdin. Headless agents get an
  Authelia single-factor exception scoped by source address, group and target.
  Logs are shipped to the host journal, not kept in containers.
- **D3 preflight:** a project's needs are read from its files (`Cargo.toml`,
  `pyproject.toml`, `package.json`) or a declaration in the repository.
- **D4 publishing:** standard excludes are configured in each clone, and only what
  the work produced is committed. A push hands the key over stdin to a
  short-lived `ssh-agent` on the build VM, for that one push.
- **D7:** test services use ports 18000–18099, so opening access later is a
  firewall change. The inference host is behind LiteLLM, so it is not a second
  egress exception.
- **X4:** the agent host's state dataset is ZFS, snapshotted on a schedule and on
  each approval. Tool VM templates use Terraform's `bpg/proxmox` provider with
  cloud-init. Containers restart unless stopped.
- **X5:** the local inference host is llama-swap on an RTX 4090 machine.