Merge pull request 'Propose the Development Plan' (#84) from development-plan into main

Reviewed-on: dongar/nisse#84 Reviewed-by: Don Garrett <dgarrett@acm.org>
2026-09-30 21:07:39 +00:00
commit 890708594b
@@ -1,11 +1,10 @@
 # Nisse — Architecture
-**Status: draft for review.** This document is to replace the architecture parts of
+**Status: agreed (PR #83).** How and in what order it gets built is in
-`docs/Architecture-Build-Spec.md`. How and in what order it gets built will go in a
+`docs/Development-Plan.md`. Until that plan's first milestone retires
-separate document, `docs/Development-Plan.md`, written once this one is agreed.
+`docs/Architecture-Build-Spec.md`, code comments still cite the old spec's section
 Until then the old spec stays in place, and code comments still cite its section
 numbers (A1–A18, B, C, F). This document uses different letters (P, L, D, O, X, Q)
-so the two can't be confused while the references are updated.
+so the two can't be confused meanwhile.
 This document says what Nisse is, how its parts fit together, and why. It records
 decisions and the reasons for them. It doesn't record status or sequencing.
@@ -788,6 +787,8 @@ milestone's marking.
  opening access later is a firewall change.
 - **Test containers on an internal Docker network with no route out.**
 - **Build storage on a separate mount** from the state volume.
 - **A browser** (a shared Playwright tool, headless) **reaches only the internal
  test network,** to exercise web apps the agent built and reproduce bugs. No SSO.
 ---
@@ -0,0 +1,333 @@
 # Nisse — Development Plan
 **Status: proposed for discussion.** How to get from the code as it is to the full
 architecture in `docs/Architecture.md`, in what order, and how each step is
 accepted. The architecture says *what and why*; this document says *how and when*,
 and is revised as work lands.
 It follows the format the architecture asks agents to use (D2): phases of
 milestones, each with a goal, scope and acceptance criteria, and each marked
 **pre-approved** or **needs discussion**.
 ---
 ## 1. Where we are
 A survey of the code on `main` (2026-09-30) against each architecture section.
 **Done** means it exists and matches the architecture; **partial** means something
 exists but needs changing; **none** means not started.
 Only `nisse-dev` has real code (about 10,000 lines in `nisse-dev/src/nisse_dev/`).
 `nisse-core` and `nisse-ops` are placeholder packages.
 | Section | Status | Notes |
 |---|---|---|
 | **P** Principles | — | Guidance; each milestone below applies them |
 | **L1** Work model | partial | One work item at a time with a minted ID; but the state store is a single row (`agent_state`, `CHECK (id = 1)`), not a work-items table with an active pointer. No duties, parking or switching (only `nisse-ops` needs duties) |
 | **L2** Agent loop | partial | `control.py`: a fixed state machine (arch → milestone gates → executing → PR open), with the no-stranding rule, `stop`, `abandon`, restart questions. To be replaced by the conversational loop |
 | **L3** Authority | partial | Revision binding and arrival stamping are done (`gates.py`, #74). Approvals are gates, not permissions; no pre-approved milestones, no trust dial |
 | **L4** Action classifier | none | |
 | **L5** Conversation | partial | Matrix transport, mention handling, case-insensitive commands. Only the planner can ask (`ask_human`); no question handling, no explanations or progress prose, a misleading reply in `IDLE` |
 | **L6** Context and memory | partial | Context fitting is done (`context.py`). The transcript isn't scoped per work item and has no attribution or orchestrator facts (#78). No scope or agent memory |
 | **L7** Canonical log | partial | `canonical.py`: user, assistant, tool call/result, plan, gate, delegation, compaction events. Missing: work item and actor on every event, `fact`, `classifier_verdict`, `knowledge_retrieved` |
 | **L8** Durability and recovery | mostly done | WAL store, intents, boot reconciliation, adopting a published PR (#75), comment cursors and redelivery (#68, #76), Quint model (`specs/`). Needs moving onto the new loop and the work-items schema |
 | **L9** Models, roles, helpers | partial | LiteLLM client and catalog, context budgets, delegation with passthrough and caps, `ask_expert`, tool-choice handling, eval capture. Roles are executor/planner/delegate/consultant with a frontier planner; no `driver`, no capability profiles, no driver-chosen escalation |
 | **L10** Knowledge store | none | |
 | **L11** Prompt injection | partial | Operator allowlists on every channel; no classifier, no structured intake beyond Gitea's JSON |
 | **L12** Secrets, identity, logging | partial | Structured state-change log (`oplog.py`); secrets from `.env` files; no Vaultwarden, no LLDAP identity |
 | **D2** Documents as workflow | none | Plans are posted to a Gitea issue; no project `ARCHITECTURE.md` or build plans in the repo |
 | **D3** Workspace | partial | Cloned once and reused as-is; no fetch, no fresh workspace, no preflight (#62, #27) |
 | **D4** Git and PRs | partial | Idempotent publishing (#75), committed work published (#63); `git add -A` sweeps in artifacts (#80); branches cut at publish from whatever `HEAD` is (#62) |
 | **D5** Review and merge | partial | Review loop on PR conversation comments; inline comments unseen (#69); watch stall (#70); operator merges; no trust dial (#65) |
 | **D7** Deployment | none | Bring-up runs on this box as root; no VM, VLAN or egress policy |
 | **O** `nisse-ops` | none | |
 | **X** Deployment model | none | |
 ---
 ## 2. The environment today
 **Supporting services** (X5) are installed and running: Matrix (Synapse), Gitea,
 LiteLLM, Vikunja, Outline, Vaultwarden, LLDAP with Authelia. Per-agent accounts and
 tokens are created as each agent needs them. So far `nisse-dev` has a Matrix test
 identity and room, a Gitea test account and sandbox repo, and dedicated LiteLLM
 keys.
 **Local inference:** an RTX 4090 host, `inference-4090.garrett-hall.com`, running
 llama-swap (one model resident at a time), capped at 320 W. The executor benchmark
 (PR #55) chose `qwen3.8-27b` for `nisse-dev`'s local work.
 **Models by role today** (`nisse-dev.example.yaml`): executor `qwen3.8-27b`, planner
 and delegate `claude-opus-5-5`, consultant `claude-haiku-4-5`. Phase 2 moves to the
 architecture's roles (L9): driver local, helpers frontier.
 **Bring-up runs** happen on this box, against test resources: the sandbox repo, the
 test Matrix room, a budget-capped LiteLLM key; configuration in `~/nisse-e2e/`.
 ---
 ## 3. How we work
 - **One PR per milestone,** usually. Every PR still starts with a short
  implementation plan per `CLAUDE.md`. For a **pre-approved** milestone that plan is
  a quick confirmation; a **needs discussion** milestone starts with a design
  discussion before any plan.
 - **Every change comes with tests,** per `CLAUDE.md`, and runs through
  `scripts/check.sh` (lint, format, unit and integration tests).
 - **The Quint model moves with the loop.** Any change to how the loop enters or
  leaves a waiting point updates `specs/`, and every counterexample is reproduced
  against the real code before it is fixed.
 - **A bring-up run on this box ends each phase** where it can be run live. What it
  finds is filed as issues and fixed before the next phase starts, or explicitly
  deferred.
 - **Bring-up guards on this box** (it isn't the sealed VM): the agent runs as an
  unprivileged user, touches only test resources, and keeps today's structural
  gates on in its bring-up configuration until the classifier and trust dial are
  proven (Phase 3).
 - **This plan is revised as we go:** a milestone's scope may move once work shows
  its real shape, recorded here in the same PR.
 ---
 ## 4. Phases
 ### Phase 0 — Housekeeping
 **M0.1 Retire the old spec.** *Pre-approved.*
 - Update every section reference in code, tests, `specs/` and the `CLAUDE.md` and
  `README.md` files from the old spec's A–H to the new letters (table in §6).
 - Delete `docs/Architecture-Build-Spec.md` (git keeps it); drop §6 from this plan.
 - Acceptance: `grep` finds no reference to the old spec or its section numbers.
 **M0.2 A bring-up script, and an unprivileged agent user on this box.** *Pre-approved.*
 - One script: set aside the previous run's state, workspace and log; start the agent
  as an unprivileged user; collect logs and the transcript afterwards.
 - Acceptance: a smoke run starts, answers `status`, and stops cleanly as that user.
 ### Phase 1 — Foundations the new loop needs
 These fix problems found in the live runs that **survive the loop change.** Issues
 that only exist because of `control.py`'s state machine (#61, #79, #81) are folded
 into Phase 2's design instead of being fixed twice.
 **M1.1 Work items in the state store.** *Needs discussion* (schema).
 - A work-items table with an active pointer (L1); every canonical event carries its
  work item and its actor (L7). A migration for existing stores.
 - Acceptance: two work items back to back keep separate state and events; a
  restart keeps both; migration tests.
 **M1.2 Transcript views.** *Needs discussion* (rendering).
 - Per-role rendering (L6): a role sees only its work item, attributed entries, and
  orchestrator `fact` events; never another role's raw turns as its own (#78).
 - Acceptance: the second live run's confusion can't happen (a planning call after
  a milestone contains the milestone fact and PR, and no executor turns).
 **M1.3 A fresh workspace at `use`, and harness-owned branches.** *Pre-approved.*
 - Fetch; clean checkout of the latest base; set aside anything left over and report
  it (D3). Milestone branches cut by the harness from the task's workspace;
  "changes to publish" counted from the milestone's start (D4). Covers #62 and the
  clone-on-use half of #27.
 - Acceptance: a leftover branch, uncommitted change and stale `origin` are each
  handled and reported; a second milestone's PR contains only its own commits.
 **M1.4 Publish only the work.** *Pre-approved.*
 - Standard excludes in the clone, and commit only what the work changed (D4, #80).
 - Acceptance: a milestone that ran pytest publishes no `.pyc`; excluded files are
  reported.
 **M1.5 Review intake.** *Pre-approved.*
 - Inline review comments and review bodies (#69); no stall when a watch is
  replaced from its own callback (#70); replies in the thread where the comment was
  made (D5).
 - Acceptance: tests for each; inline comments work in the phase's bring-up run.
 **Phase 1 acceptance:** a bring-up run on the current loop with two milestones and
 a review comment, including inline ones.
 ### Phase 2 — The conversational loop
 Built **alongside** `control.py`, reusing the state store, canonical log, context
 fitting, publisher, pollers and recovery; `nisse-dev` switches over when the new
 loop passes its tests and a bring-up run, and `control.py` is retired.
 **M2.1 Loop design and its model.** *Needs discussion.*
 - A short design note (in the PR, then folded into the architecture where it
  settles something): the loop's turns, waiting points, pending questions,
  permissions it checks, and recovery. A new Quint model for it before code, with
  the no-stranding, approval-binding and at-least-once properties carried over.
 **M2.2 The loop core.** *Needs discussion.*
 - The driver-driven turn loop (L2): conversation, tool calls, pending questions,
  `stop`, `abandon`, intents and boot recovery on the work-items schema.
 - **Questions never change state** (L5): the recognition rule is decided here (Q).
 - Acceptance: the new Quint model's scenarios reproduced as tests; no-stranding
  under injected faults, as `test_no_stranding.py` does today.
 **M2.3 Roles and capability profiles.** *Pre-approved.*
 - Roles become `driver`, `helper`, `expert`, `classifier` (L9); capability profiles
  in configuration, filled from the benchmark; the driver's prompt encourages asking
  for help; observable escalation triggers; delegation and `ask_expert` wired to the
  new loop. Supersedes #46 (separate architect and planner roles).
 **M2.4 Conversation.** *Pre-approved* (the behaviour is set by L5).
 - Explanations with plans and revisions, finished-work summaries, progress in prose
  (#42), handling messages that can't be acted on yet, asking mid-work (#77).
 **M2.5 Documents as the workflow.** *Needs discussion.*
 - Project `ARCHITECTURE.md` and build plans in the repo, discussed and landed by PR;
  milestones marked pre-approved or needs discussion and whether they change code;
  permissions derived from them (L3, D2).
 - Milestones and projects end properly: the plan declares its milestones; a
  no-change milestone that wasn't meant to change code is done; moving on checks the
  PR's state; finishing closes the issue (D4, D5). Supersedes #61, #79, #81.
 **M2.6 Switch over and retire `control.py`.** *Pre-approved, after acceptance.*
 - `nisse-dev` runs the new loop; `control.py`, its tests and the old Quint model are
  removed.
 **Phase 2 acceptance:** a bring-up run: a new small project from conversation to an
 agreed `ARCHITECTURE.md` and build plan, two milestones (one pre-approved), a review
 comment answered, a question asked by the agent mid-milestone, and a crash
 mid-review.
 ### Phase 3 — Authority
 **M3.1 Action classes and the fast path.** *Needs discussion.*
 - Every tool action classified by effect (P3); static fast-path rules per agent
  (D6); the scope each class-2 action is checked against.
 **M3.2 The action classifier.** *Needs discussion.*
 - L4, including the model choice (Q), fail-closed behaviour, logged verdicts, and
  operator overrides as an evaluation set.
 **M3.3 Permissions and the trust dial.** *Needs discussion.*
 - Approvals as scoped permissions, the operator-set trust dial, and agent merging
  behind branch protection the agent can't satisfy, verified by a test (L3, D4, D5,
  #65).
 **Phase 3 acceptance:** a bring-up run with the classifier active and gates first
 on, then deliberately loosened in the bring-up configuration.
 ### Phase 4 — The environment
 **M4.1 Toolchain preflight.** *Pre-approved.* Detect, check, report before planning;
 toolchain facts in the transcript (D3).
 **M4.2 Providing toolchains.** *Needs discussion.* Decide dev containers vs the VM
 image (Q), then build it.
 **M4.3 Environment repair.** *Pre-approved.* The driver may clean caches and
 reinstall project dependencies within a task, and reports it (D3).
 ### Phase 5 — Memory and knowledge
 **M5.1 The Outline client.** *Pre-approved.* Per-agent Outline user and token,
 collections and documents by ID, revision reads, one writer per document (L10).
 **M5.2 Agent memory.** *Needs discussion.* When notes are written and read, the
 index, verbatim corrections (L6).
 **M5.3 Scope memory for repositories.** *Pre-approved.* Project documents read at
 work-item start and snapshotted; summaries at close (L6).
 ### Phase 6 — `nisse-dev` in production
 **M6.1 The VM, VLAN and egress policy.** *Needs discussion.* D7, X1, X2.
 **M6.2 Secrets and identity.** *Pre-approved.* Vaultwarden machine accounts, LLDAP
 principal, secrets out of `.env` files (L12).
 **M6.3 Images and delivery.** *Pre-approved.* Gitea Actions → OCI registry →
 Watchtower, restarts gated on an idle flag (X4).
 **M6.4 The browser tool.** *Pre-approved.* Playwright on the internal test network
 only (D7).
 ### Phase 7 — `nisse-core` and `nisse-ops`
 **M7.1 Extract `nisse-core`.** *Pre-approved.* A package split within the monorepo:
 what both agents use moves into `nisse-core`.
 **M7.2 MCP servers.** *Needs discussion.* Per (service, site, privilege) instances,
 tool profiles, configuration (X3, #37).
 **M7.3 Approved step sequences.** *Needs discussion.* The ops plan format, matcher,
 policy file, expiry (O2).
 **M7.4 Tickets and duties.** *Needs discussion.* Vikunja intake and approvals,
 scheduled checks and alert intake as duties (O5, #33).
 **M7.5 `nisse-ops` deployment.** *Needs discussion.* O6.
 ### Not yet scheduled
 From the architecture's open questions (Q) and older issues, to be placed when
 their trigger arrives:
 - more than one active work item per agent;
 - vision for 3D design and video work;
 - one local model for every agent, or agents taking turns on the inference host;
 - several forges and projects (#26); a Matrix room per project (#28); creating a
  project from scratch and who owns it (#29); one agent instance per project (#30);
 - infrastructure as code for agent VMs.
 ---
 ## 5. Open issues and where they land
 All 19 open issues, as of 2026-09-30:
 - **11 land in a milestone** and stay open until it is done.
 - **4 are superseded** by the new loop's design. They only exist because of
  `control.py`'s state machine, so fixing them there would be thrown away. They are
  closed, with a pointer to their milestone, when this plan is agreed.
 - **4 are not yet scheduled** and stay open until their trigger arrives.
 | Issue | Status | Lands in |
 |---|---|---|
 | #26 multiple forges | Not yet scheduled | |
 | #27 per-project state, clone on use | Milestone | M1.1 (state), M1.3 (clone) |
 | #28 room per project | Not yet scheduled | |
 | #29 create a project from scratch | Not yet scheduled | Ownership decision first |
 | #30 agent instance per project | Not yet scheduled | |
 | #33 Vikunja tickets | Milestone | M7.4 |
 | #37 MCP servers | Milestone | M7.2 |
 | #42 per-turn commentary | Milestone | M2.4 |
 | #46 architect and planner roles | **Superseded** | M2.3 (one driver, helpers) |
 | #61 simpler milestone gating | **Superseded** | M2.5 |
 | #62 branch at execution start | Milestone | M1.3 |
 | #65 agent merges approved PRs | Milestone | M3.3 |
 | #69 inline review comments | Milestone | M1.5 |
 | #70 watch stall | Milestone | M1.5 |
 | #77 conversational agent | Milestone | M2.2 (questions), M2.4 |
 | #78 transcript scoping and attribution | Milestone | M1.1, M1.2 |
 | #79 ending projects and milestones | **Superseded** | M2.5 |
 | #80 build artifacts in PRs | Milestone | M1.4 |
 | #81 no-change milestones | **Superseded** | M2.5 |
 ---
 ## 6. Old section numbers → new (for M0.1, then removed)
 | Old (`Architecture-Build-Spec.md`) | New (`Architecture.md`) |
 |---|---|
 | 0 Naming | Naming |
 | 0.1 What we're building, guiding principles | P1, P3–P8 |
 | 0.2 Work model | L1 |
 | A1 Transport | L5 |
 | A2 State store, A3 Boot reconciliation | L8 |
 | A4 Control loop and gating | L2 (loop, `stop`, `abandon`), L3 (approvals) |
 | A5 LLM access | L9 |
 | A6 Browser tool | D7, O4 |
 | A7 Secrets, A8 SSO identity, A9 Logging | L12 |
 | A10 Canonical transcript | L7 (schema), L6 (context) |
 | A11 Roles, A12 Escalation, A13 Delegation, A14 `ask_expert`, A15 Local accommodations, A16 Eval capture | L9 |
 | A17 Knowledge store | L10 (store), L6 (memory) |
 | A18 Prompt injection | L11 |
 | B1 Workflow | L2, D2 |
 | B2 Requirements | D2, D4, D5 |
 | B3 Deployment | D7 |
 | C1–C5 Sysadmin Agent | O1–O6 (C2 → O2, C3 → O4, C4 → O5, C5 → O6) |
 | D Knowledge (Outline) | L10 |
 | E Supporting services | X5, and §2 here |
 | F1–F4 Deployment model | X1–X4 |
 | G Build order | This plan |
 | H Open decisions | Q |