nisse/docs/Development-Plan.md

# Nisse — Development Plan

**Status: proposed for discussion.** How to get from the code as it is to the full
architecture in `docs/Architecture.md`, in what order, and how each step is
accepted. The architecture says *what and why*; this document says *how and when*,
and is revised as work lands.

It follows the format the architecture asks agents to use (D2): phases of
milestones, each with a goal, scope and acceptance criteria, and each marked
**pre-approved** or **needs discussion**.

---

## 1. Where we are

A survey of the code on `main` (2026-09-30) against each architecture section.
**Done** means it exists and matches the architecture; **partial** means something
exists but needs changing; **none** means not started.

Only `nisse-dev` has real code (about 10,000 lines in `nisse-dev/src/nisse_dev/`).
`nisse-core` and `nisse-ops` are placeholder packages.

| Section | Status | Notes |
|---|---|---|
| **P** Principles | — | Guidance; each milestone below applies them |
| **L1** Work model | partial | One work item at a time with a minted ID; but the state store is a single row (`agent_state`, `CHECK (id = 1)`), not a work-items table with an active pointer. No duties, parking or switching (only `nisse-ops` needs duties) |
| **L2** Agent loop | partial | `control.py`: a fixed state machine (arch → milestone gates → executing → PR open), with the no-stranding rule, `stop`, `abandon`, restart questions. To be replaced by the conversational loop |
| **L3** Authority | partial | Revision binding and arrival stamping are done (`gates.py`, #74). Approvals are gates, not permissions; no pre-approved milestones, no trust dial |
| **L4** Action classifier | none | |
| **L5** Conversation | partial | Matrix transport, mention handling, case-insensitive commands. Only the planner can ask (`ask_human`); no question handling, no explanations or progress prose, a misleading reply in `IDLE` |
| **L6** Context and memory | partial | Context fitting is done (`context.py`). The transcript isn't scoped per work item and has no attribution or orchestrator facts (#78). No scope or agent memory |
| **L7** Canonical log | partial | `canonical.py`: user, assistant, tool call/result, plan, gate, delegation, compaction events. Missing: work item and actor on every event, `fact`, `classifier_verdict`, `knowledge_retrieved` |
| **L8** Durability and recovery | mostly done | WAL store, intents, boot reconciliation, adopting a published PR (#75), comment cursors and redelivery (#68, #76), Quint model (`specs/`). Needs moving onto the new loop and the work-items schema |
| **L9** Models, roles, helpers | partial | LiteLLM client and catalog, context budgets, delegation with passthrough and caps, `ask_expert`, tool-choice handling, eval capture. Roles are executor/planner/delegate/consultant with a frontier planner; no `driver`, no capability profiles, no driver-chosen escalation |
| **L10** Knowledge store | none | |
| **L11** Prompt injection | partial | Operator allowlists on every channel; no classifier, no structured intake beyond Gitea's JSON |
| **L12** Secrets, identity, logging | partial | Structured state-change log (`oplog.py`); secrets from `.env` files; no Vaultwarden, no LLDAP identity |
| **D2** Documents as workflow | none | Plans are posted to a Gitea issue; no project `ARCHITECTURE.md` or build plans in the repo |
| **D3** Workspace | partial | Cloned once and reused as-is; no fetch, no fresh workspace, no preflight (#62, #27) |
| **D4** Git and PRs | partial | Idempotent publishing (#75), committed work published (#63); `git add -A` sweeps in artifacts (#80); branches cut at publish from whatever `HEAD` is (#62) |
| **D5** Review and merge | partial | Review loop on PR conversation comments; inline comments unseen (#69); watch stall (#70); operator merges; no trust dial (#65) |
| **D7** Deployment | none | Bring-up runs on this box as root; no VM, VLAN or egress policy |
| **O** `nisse-ops` | none | |
| **X** Deployment model | none | |

---

## 2. The environment today

**Supporting services** (X5) are installed and running: Matrix (Synapse), Gitea,
LiteLLM, Vikunja, Outline, Vaultwarden, LLDAP with Authelia. Per-agent accounts and
tokens are created as each agent needs them. So far `nisse-dev` has a Matrix test
identity and room, a Gitea test account and sandbox repo, and dedicated LiteLLM
keys.

**Local inference:** an RTX 4090 host, `inference-4090.garrett-hall.com`, running
llama-swap (one model resident at a time), capped at 320 W. The executor benchmark
(PR #55) chose `qwen3.8-27b` for `nisse-dev`'s local work.

**Models by role today** (`nisse-dev.example.yaml`): executor `qwen3.8-27b`, planner
and delegate `claude-opus-5-5`, consultant `claude-haiku-4-5`. Phase 2 moves to the
architecture's roles (L9): driver local, helpers frontier.

**Bring-up runs** happen on this box, against test resources: the sandbox repo, the
test Matrix room, a budget-capped LiteLLM key; configuration in `~/nisse-e2e/`.

---

## 3. How we work

- **One PR per milestone,** usually. Every PR still starts with a short
  implementation plan per `CLAUDE.md`. For a **pre-approved** milestone that plan is
  a quick confirmation; a **needs discussion** milestone starts with a design
  discussion before any plan.
- **Every change comes with tests,** per `CLAUDE.md`, and runs through
  `scripts/check.sh` (lint, format, unit and integration tests).
- **The Quint model moves with the loop.** Any change to how the loop enters or
  leaves a waiting point updates `specs/`, and every counterexample is reproduced
  against the real code before it is fixed.
- **A bring-up run on this box ends each phase** where it can be run live. What it
  finds is filed as issues and fixed before the next phase starts, or explicitly
  deferred.
- **Bring-up runs on this development VM,** as root, like production. Guards: it
  touches only test resources, and keeps today's structural gates on in its
  bring-up configuration until the classifier and trust dial are proven (Phase 3).
- **This plan is revised as we go:** a milestone's scope may move once work shows
  its real shape, recorded here in the same PR.

---

## 4. Phases

### Phase 0 — Housekeeping

**M0.1 Retire the old spec.** *Pre-approved.* **Done.**
- Update every section reference in code, tests, `specs/` and the `CLAUDE.md` and
  `README.md` files from the old spec's A–H to the new letters.
- Delete `docs/Architecture-Build-Spec.md` (git keeps it), and the old-to-new
  section table this plan carried for the purpose.
- Acceptance: `grep` finds no reference to the old spec or its section numbers.

**M0.2 A bring-up script.** *Pre-approved.* **Done.**
- `scripts/bringup.sh`: set aside the previous run's state, workspace and log;
  export the ref under test; start the agent as a systemd unit, so it outlives the
  shell that started it; collect the log and a readable transcript
  (`python -m nisse_dev.dump`) afterwards.
- The agent runs as root on this development VM, as it will on its own VM in
  production (D7), so installing packages works as it will there.
- Acceptance: a smoke run starts, answers `status`, stops cleanly, and `collect`
  produces the log and transcript.

### Phase 1 — Foundations the new loop needs

These fix problems found in the live runs that **survive the loop change.** Issues
that only exist because of `control.py`'s state machine (#61, #79, #81) are folded
into Phase 2's design instead of being fixed twice.

**M1.1 Work items in the state store.** *Needs discussion* (schema). **Done.**
- A work-items table with an active pointer (L1); every canonical event carries its
  work item and its actor (L7). A migration for existing stores.
- Acceptance: two work items back to back keep separate state and events; a
  restart keeps both; migration tests.

**M1.2 Transcript views.** *Needs discussion* (rendering). **Done.**
- Per-role rendering (L6): a role sees only its work item, attributed entries, and
  orchestrator `fact` events; never another role's raw turns as its own (#78).
- Acceptance: the second live run's confusion can't happen (a planning call after
  a milestone contains the milestone fact and PR, and no executor turns).

**M1.3 A fresh workspace at `use`, and harness-owned branches.** *Pre-approved.*
- Fetch; clean checkout of the latest base; set aside anything left over and report
  it (D3). Milestone branches cut by the harness from the task's workspace;
  "changes to publish" counted from the milestone's start (D4). Covers #62 and the
  clone-on-use half of #27.
- Acceptance: a leftover branch, uncommitted change and stale `origin` are each
  handled and reported; a second milestone's PR contains only its own commits.

**M1.4 Publish only the work.** *Pre-approved.*
- Standard excludes in the clone, and commit only what the work changed (D4, #80).
- Acceptance: a milestone that ran pytest publishes no `.pyc`; excluded files are
  reported.

**M1.5 Review intake.** *Pre-approved.*
- Inline review comments and review bodies (#69); no stall when a watch is
  replaced from its own callback (#70); replies in the thread where the comment was
  made (D5).
- Acceptance: tests for each; inline comments work in the phase's bring-up run.

**Phase 1 acceptance:** a bring-up run on the current loop with two milestones and
a review comment, including inline ones.

### Phase 2 — The conversational loop

Built **alongside** `control.py`, reusing the state store, canonical log, context
fitting, publisher, pollers and recovery; `nisse-dev` switches over when the new
loop passes its tests and a bring-up run, and `control.py` is retired.

**M2.1 Loop design and its model.** *Needs discussion.*
- A short design note (in the PR, then folded into the architecture where it
  settles something): the loop's turns, waiting points, pending questions,
  permissions it checks, and recovery. A new Quint model for it before code, with
  the no-stranding, approval-binding and at-least-once properties carried over.

**M2.2 The loop core.** *Needs discussion.*
- The driver-driven turn loop (L2): conversation, tool calls, pending questions,
  `stop`, `abandon`, intents and boot recovery on the work-items schema.
- **Questions never change state** (L5): the recognition rule is decided here (Q).
- Acceptance: the new Quint model's scenarios reproduced as tests; no-stranding
  under injected faults, as `test_no_stranding.py` does today.

**M2.3 Roles and capability profiles.** *Pre-approved.*
- Roles become `driver`, `helper`, `expert`, `classifier` (L9); capability profiles
  in configuration, filled from the benchmark; the driver's prompt encourages asking
  for help; observable escalation triggers; delegation and `ask_expert` wired to the
  new loop. Supersedes #46 (separate architect and planner roles).

**M2.4 Conversation.** *Pre-approved* (the behaviour is set by L5).
- Explanations with plans and revisions, finished-work summaries, progress in prose
  (#42), handling messages that can't be acted on yet, asking mid-work (#77).

**M2.5 Documents as the workflow.** *Needs discussion.*
- Project `ARCHITECTURE.md` and build plans in the repo, discussed and landed by PR;
  milestones marked pre-approved or needs discussion and whether they change code;
  permissions derived from them (L3, D2).
- Milestones and projects end properly: the plan declares its milestones; a
  no-change milestone that wasn't meant to change code is done; moving on checks the
  PR's state; finishing closes the issue (D4, D5). Supersedes #61, #79, #81.

**M2.6 Switch over and retire `control.py`.** *Pre-approved, after acceptance.*
- `nisse-dev` runs the new loop; `control.py`, its tests and the old Quint model are
  removed.

**Phase 2 acceptance:** a bring-up run: a new small project from conversation to an
agreed `ARCHITECTURE.md` and build plan, two milestones (one pre-approved), a review
comment answered, a question asked by the agent mid-milestone, and a crash
mid-review.

### Phase 3 — Authority

**M3.1 Action classes and the fast path.** *Needs discussion.*
- Every tool action classified by effect (P3); static fast-path rules per agent
  (D6); the scope each class-2 action is checked against.

**M3.2 The action classifier.** *Needs discussion.*
- L4, including the model choice (Q), fail-closed behaviour, logged verdicts, and
  operator overrides as an evaluation set.

**M3.3 Permissions and the trust dial.** *Needs discussion.*
- Approvals as scoped permissions, the operator-set trust dial, and agent merging
  behind branch protection the agent can't satisfy, verified by a test (L3, D4, D5,
  #65).

**Phase 3 acceptance:** a bring-up run with the classifier active and gates first
on, then deliberately loosened in the bring-up configuration.

### Phase 4 — The environment

**M4.1 Toolchain preflight.** *Pre-approved.* Detect what the project needs, check
it, and install what's missing before planning; toolchain facts in the transcript
(D3).

**M4.2 Recording installs.** *Pre-approved.* Every package or toolchain the agent
installs is recorded as a fact and in the project's scope memory, so the
environment can be rebuilt; recurring ones are noted for the VM image (D3, X4).

**M4.3 Environment repair.** *Pre-approved.* The driver may clean caches and
reinstall project dependencies within a task, and reports it (D3).

### Phase 5 — Memory and knowledge

**M5.1 The Outline client.** *Pre-approved.* Per-agent Outline user and token,
collections and documents by ID, revision reads, one writer per document (L10).

**M5.2 Agent memory.** *Needs discussion.* When notes are written and read, the
index, verbatim corrections (L6).

**M5.3 Scope memory for repositories.** *Pre-approved.* Project documents read at
work-item start and snapshotted; summaries at close (L6).

### Phase 6 — `nisse-dev` in production

**M6.1 The VM, VLAN and egress policy.** *Needs discussion.* D7, X1, X2.

**M6.2 Secrets and identity.** *Pre-approved.* Vaultwarden machine accounts, LLDAP
principal, secrets out of `.env` files (L12).

**M6.3 Images and delivery.** *Pre-approved.* Gitea Actions → OCI registry →
Watchtower, restarts gated on an idle flag (X4).

**M6.4 The browser tool.** *Pre-approved.* Playwright on the internal test network
only (D7).

### Phase 7 — `nisse-core` and `nisse-ops`

**M7.1 Extract `nisse-core`.** *Pre-approved.* A package split within the monorepo:
what both agents use moves into `nisse-core`.

**M7.2 MCP servers.** *Needs discussion.* Per (service, site, privilege) instances,
tool profiles, configuration (X3, #37).

**M7.3 Approved step sequences.** *Needs discussion.* The ops plan format, matcher,
policy file, expiry (O2).

**M7.4 Tickets and duties.** *Needs discussion.* Vikunja intake and approvals,
scheduled checks and alert intake as duties (O5, #33).

**M7.5 `nisse-ops` deployment.** *Needs discussion.* O6.

### Not yet scheduled

From the architecture's open questions (Q) and older issues, to be placed when
their trigger arrives:
- more than one active work item per agent;
- vision for 3D design and video work;
- one local model for every agent, or agents taking turns on the inference host;
- several forges and projects (#26); a Matrix room per project (#28); creating a
  project from scratch and who owns it (#29); one agent instance per project (#30);
- infrastructure as code for agent VMs.

---

## 5. Open issues and where they land

All 19 open issues, as of 2026-09-30:
- **11 land in a milestone** and stay open until it is done.
- **4 are superseded** by the new loop's design. They only exist because of
  `control.py`'s state machine, so fixing them there would be thrown away. They are
  closed, with a pointer to their milestone, when this plan is agreed.
- **4 are not yet scheduled** and stay open until their trigger arrives.

| Issue | Status | Lands in |
|---|---|---|
| #26 multiple forges | Not yet scheduled | |
| #27 per-project state, clone on use | Milestone | M1.1 (state), M1.3 (clone) |
| #28 room per project | Not yet scheduled | |
| #29 create a project from scratch | Not yet scheduled | Ownership decision first |
| #30 agent instance per project | Not yet scheduled | |
| #33 Vikunja tickets | Milestone | M7.4 |
| #37 MCP servers | Milestone | M7.2 |
| #42 per-turn commentary | Milestone | M2.4 |
| #46 architect and planner roles | **Superseded** | M2.3 (one driver, helpers) |
| #61 simpler milestone gating | **Superseded** | M2.5 |
| #62 branch at execution start | Milestone | M1.3 |
| #65 agent merges approved PRs | Milestone | M3.3 |
| #69 inline review comments | Milestone | M1.5 |
| #70 watch stall | Milestone | M1.5 |
| #77 conversational agent | Milestone | M2.2 (questions), M2.4 |
| #78 transcript scoping and attribution | Milestone | M1.1, M1.2 |
| #79 ending projects and milestones | **Superseded** | M2.5 |
| #80 build artifacts in PRs | Milestone | M1.4 |
| #81 no-change milestones | **Superseded** | M2.5 |