diff --git a/docs/Architecture-Build-Spec.md b/docs/Architecture-Build-Spec.md index c39387e..7ae8ea3 100644 --- a/docs/Architecture-Build-Spec.md +++ b/docs/Architecture-Build-Spec.md @@ -159,6 +159,17 @@ come back to something. Components throughout this document refer to them. arrive is a safe point, with one exception: switching is refused while intents are unresolved (A2). That keeps every pending intent owned by the active work item, so boot reconciliation (A3) stays single-item. +- **The operator can always switch; the agent may switch on its own in one case.** + When the active work item is waiting on the operator (an open gate, a pending + question, or a halt in `AWAIT_INPUT`) and another work item is ready, the agent + switches to it, oldest first, and posts a notice naming both items. An alert + must not sit for hours behind an unanswered prompt. The agent never switches + away from an item that is working. The item it left stays waiting; switching + answers nothing on its behalf. +- **Parking gives up authority.** Parking a work item unmounts any elevated MCP + instances (F3) its approved plan mounted. They are remounted when it becomes + active again, if its approval has not expired (C2). An approved plan's + credentials never serve another item. - **Nothing is lost while parked.** Intake for a parked work item, such as comments on its Gitea issue or PR, is read from a durable per-thread cursor when it becomes active again. It is not dropped, and it is not replayed from the beginning. @@ -244,10 +255,18 @@ Every startup (clean or post-crash), before accepting operator commands: (tree diff for edits, Gitea query for pushes), or on any branch/log mismatch. The judgment is an observable check, never model self-assessment — the same reasoning as A12. -6. Any other interruption → move to **`AWAIT_INPUT`** and post to transport +6. **Interrupted mid-sequence (C2) — resume only on evidence.** An approved step + sequence may interrupt itself: rebooting the host the agent runs on is a + legitimate step. On restart, **continue** the sequence only when the intent log + names exactly which step was in flight and that step's expected outcome checks + out against the live system, and the approval has not expired. Anything else + (no recorded outcome check, a failed check, or more than one step + possibly in flight) gates to `AWAIT_INPUT`. The step counts as completed only + by that check, never by assuming it ran. +7. Any other interruption → move to **`AWAIT_INPUT`** and post to transport (e.g. "restarted mid-milestone-2, branch at commit abc123, continue or re-plan?"). Never blind-resume. -7. Otherwise resume the last committed state. +8. Otherwise resume the last committed state. ### A4. Control Loop, State Machine, and Gating Primitives @@ -289,11 +308,20 @@ Requirements: - **Approvals bind to a work item, too (0.2).** A bare `approve` binds to the *active* work item's latest revision, under the unambiguity rule above. An approval arriving on a *parked* item's external record (a Gitea or Vikunja - comment) is delivered when that item next becomes active. It is never applied to - the active item. + comment) is delivered when that item next becomes active. So is a Matrix reply to + a parked item's plan or prompt: the reply relation binds it to that item. Neither + is ever applied to the active item. - **Autonomy inside an approved unit of work:** all tool/command use auto-approved (Claude Agent SDK `bypassPermissions` or an allow-all `canUseTool`). Escalation to - the human happens *only* via `ask_human`. + the human happens *only* via `ask_human`. For the Sysadmin Agent, "approved" + means the step list of C2. Only calls that match an approved step are + auto-approved. +- **Kill switch.** The operator command `stop` works in every state, for every + agent. It ends the current turn or delegation at the next tool boundary, unmounts + elevated MCP instances (F3), and moves to `AWAIT_INPUT`. The unfinished delegation + or step sequence is then handled like any other halt: continue, re-plan, or + abandon, on the operator's word. Unattended work needs a way to end it that does + not depend on the model cooperating. - The loop must survive restart mid-work (relies on A2/A3). ### A5. LLM Access *(tiered)* @@ -642,33 +670,24 @@ Part D describes the service; this defines the contract. desynchronise from the project. **Agent-owned knowledge is always a cache, never authority.** An agent may rely on -it to know *what exists and where to look*; it must confirm a specific fact -against the live system *before acting on it*. Hard requirement for anything in -C2's confirm-first tier; advisory for read-only and reversible actions. +it to know *what exists and where to look*. It must not rely on it for the facts an +action depends on. A prompt cannot enforce that: the model cannot be prevented from +*believing* a fact it has read. The enforcement is elsewhere, and it is structural: +- **Step preconditions (C2) carry the weight.** Every elevated write is an approved + step, and a step that depends on a fact states it as a precondition, checked + against the live system immediately before the step runs. A stale world model + can produce a wrong plan, but it cannot turn that plan into an action on a + system that no longer matches it. +- **Provenance at the human gate.** When a plan is drafted from recalled facts, the + approval message says so: "target recalled from the world model, observed + 2026-08-02" versus "confirmed by `zfs list` two minutes ago". The operator is + the authority there; this makes the evidence quality visible. +- **Audit for the rest.** Retrievals are logged (A10), so a wrong fact can be traced + to the actions that followed it. -**Enforcement is structural, because a prompt is not enforcement.** "Treat this as -a cache" is exactly the advisory guarantee A4 and A12 reject, and the model cannot -be prevented from *believing* a fact it has read. The enforceable goal is -narrower: **an unverified belief must not become an irreversible action.** -1. **The store decides what it serves.** Facts carry provenance and an observed-at - timestamp; retrieval marks or withholds anything past a staleness bound for - high-consequence classes. -2. **Verification as a precondition, checked against the log.** For confirm-first - actions, refuse unless a verifying observation exists *in this task's canonical - log*, recorded after the task began, naming the action's target. Inspectable - without asking the model anything — the same move as A12's observable triggers. - Approximate by nature, so build it for the confirm-first tier only. -3. **Provenance at the human gate** — the primary reliance, because C2 already - stops these actions for confirmation. The prompt states evidence quality: - "target recalled from the world model, observed 2026-08-02, not verified this - session" versus "confirmed by `zfs list` two minutes ago". The operator is - already the authority there; this makes the risk legible rather than invisible. -4. **Audit, not prevention, for the cheap tiers.** Log which facts were retrieved - and which actions followed, so a wrong fact is traceable afterwards. - -Residual risk, stated plainly: a model can retype a fact from context with no -retrieval event, and no mechanism here catches that. That is why the world model -stays out of the authority path entirely. +**Dropped:** withholding facts past a staleness bound at retrieval, and refusing +confirm-first actions without a verifying observation in the log. Both were +approximations of what step preconditions (C2) now check directly. **Never an instruction channel, and never an approval channel.** - **Task instructions never come from the knowledge store.** Work is driven by @@ -699,15 +718,14 @@ a model rewriting a precise fact loses precision invisibly. log** — the text, plus document id and revision for provenance — never referenced by handle alone: A10 forbids a provider handle being the only reference to content, and these documents are mutable. Snapshot once per task at first use, not per turn. -**A3 consequence:** a task resumed across a restart carries the old snapshot, so -boot reconciliation re-fetches snapshotted knowledge and flags divergence rather -than silently continuing on a stale copy. +A task resumed across a restart carries the old snapshot; step preconditions, not +a re-fetch, are what stop it acting on a stale copy. **Retrieval is structural first, discretionary second** — the A12/A14 pattern. -- Procedures are **bound to C2 action classes**: the policy taxonomy and the - procedure index are the same list. Binding is deterministic and auditable - ("which procedure governed this action?" has an exact answer), and costs nothing - extra because the taxonomy must exist anyway. +- Procedures are **bound to entries in the C2 policy file** (a tool, or a protected + target), and fetched when a plan that uses that entry is drafted. Binding is + deterministic and auditable ("which procedure governed this action?" has an + exact answer), and costs little because the policy file must exist anyway. - A model-invoked **search** exists *in addition* for discovery, never as a replacement for the bound retrieval — a procedure the model may skip does not constrain it. @@ -715,10 +733,10 @@ than silently continuing on a stale copy. **Outage posture is asymmetric.** Unlike B2's poller, "never fails a task" does not generalise here. - **Write failure never blocks:** retry, or record locally and reconcile. -- **Read failure for a procedure bound to a confirm-first action gates.** - Proceeding is not a delayed reaction; it is acting unconstrained on precisely the - operation someone wrote a procedure for. -- Read failure for read-only or reversible actions proceeds, logging the gap. +- **Read failure for a bound procedure gates the plan.** Planning without it is not + a delayed reaction; it is acting unconstrained on precisely the operation someone + wrote a procedure for. +- Read failure for anything else proceeds, logging the gap. **Accepted single point of failure.** Once the agent is the sole author and Outline the only home, losing Outline loses the world model. No local mirror: infrastructure @@ -726,7 +744,8 @@ facts are re-derivable by observation, which is the same premise the cache rule rests on. **Deferred:** the operator-writable procedures collection. The interface is defined -here; the content and the C2 binding list arrive with the policy taxonomy (Part H). +here; the content arrives when there are procedures worth writing, and binds to the +C2 policy file. **Concrete mapping onto Outline.** Observed in the running instance: a **collection** carries per-user permissions with three tiers — visible-to (read), read-write, and @@ -764,9 +783,9 @@ manager. Permissions are collection-scoped, not per-document. Outline can expose search, read, create and edit over MCP (remote, Streamable HTTP, enabled workspace-wide). **Not adopted for the retrieval path**, because A17's -enforcement depends on this library sitting *between* the model and the store: staleness -marking, snapshotting into the canonical log, binding procedures to action classes, and -gating a failed confirm-first read all happen to a retrieval before the model sees it. +contract depends on this library sitting *between* the model and the store: +snapshotting into the canonical log, binding procedures to C2 policy entries, and +gating a failed bound read all happen to a retrieval before the model sees it. MCP's purpose is the opposite — direct model access — and a procedure the model may decline to fetch does not constrain it (A12). @@ -778,6 +797,43 @@ membership, and that the workspace-level toggle does not force broader exposure intended. +### A18. Prompt Injection + +**Assume any model that reads untrusted content may be compromised.** There is no +robust way to tell a model "this is data, not instruction". Instructions and data +reach it as one token stream, and every marker meant to separate them is text that +the data can imitate or argue against. Role separation, delimiters and detection +classifiers lower the success rate of injection. None of them bring it to zero, and +an attacker gets to try repeatedly. So the design does not depend on the model +telling them apart. It limits what a compromised model can do. + +Untrusted content, for these agents: alert payloads, log lines, ticket and comment +text, web pages the browser reads, repository contents, and downloaded packages. + +Requirements: +- **Gates hold against a compromised model.** Nothing a model says can approve a + plan (A4), widen what a plan authorizes (C2), or reach a credential that was not + mounted (F3). Every one of those rules is enforced in code, outside the model. +- **Approval messages are rendered by code from what will actually run** (C2), + never from a model's description of it. Otherwise a compromised model can write + "restart Gitea" above a step that destroys a dataset. +- **Instruction channels are operator-scoped.** Only allowlisted identities can + drive work: operator commands (A1), PR review comments (B2), action-triggering + tickets (C4). Content from anyone else is data. +- **Parse intake into structured fields in code** wherever the source allows it, + e.g. alert name, severity, host, from Alertmanager's JSON. Less free text then + reaches the model at all, and what reaches it is labelled by where it came from. +- **Check each agent against the lethal trifecta:** access to private data, exposure + to untrusted content, and a channel to send data out. Damage needs all three. + - `nisse-dev` reads untrusted content and holds little private data: its Gitea + token and the repos it works on. Its only ways out are Gitea and package + registries (B3). The registries are an accepted, narrow residual channel. + Routing them through a local mirror would close it, if that ever becomes worth + doing. + - `nisse-ops` has all three: it reads alerts and logs, holds infrastructure + credentials, and has broad egress. It relies on the gates above, not on + removing a leg. + --- ## PART B — THE DEV AGENT / `nisse-dev` (build first) @@ -813,8 +869,8 @@ PR_OPEN - **One active project at a time** (park-and-switch, 0.2). Operator selects it; other projects stay parked at their last position and resume where they left off. - **Operator commands** (over transport): `use `, `approve`, - `reject `, `start milestone `, `status`, plus free text during - planning (forwarded to the agent as answers). + `reject `, `start milestone `, `status`, `stop` (A4), plus free + text during planning (forwarded to the agent as answers). - **Plan gating** per A4: architecture plan and each milestone plan require sign-off before the next phase. Enforced structurally. Plans and approvals are externalized to Gitea issue comments (Part H, decided); Matrix and Gitea are both approval @@ -891,7 +947,7 @@ tickets; gated on consequences. Built after the Dev Agent works. | Axis | Dev Agent | Sysadmin Agent | |---|---|---| | Rollback | revert snapshot / discard branch | none for `zfs destroy` or a firewall lockout | -| Trust model | allow-all in sandbox | tiered **allow / deny / confirm** policy | +| Trust model | allow-all in sandbox | **approved step sequences**, checked on every call | | Work shape | long milestones → a PR | short reactive tasks + scheduled checks → live changes | | Driver | Gitea projects | **Vikunja tickets** | | Placement | sealed, own VLAN, no Tailscale | Tailscale, credentials, broad reach | @@ -901,28 +957,90 @@ deployments over the same library. ### C2. Permission Policy Engine *(the defining addition)* -Tiered policy, evaluated on every action: -- **Read-only telemetry** (metrics, logs, status) — free. -- **Reversible service actions** (restart a container) — allowed within an approved - plan, without further confirmation. -- **High-consequence actions** (storage, networking, auth) — **confirm-first** over - transport before executing, even within an approved plan. - **Every write requires an approved plan first.** Write credentials exist only in the -elevated MCP server instances (F3), and those are mounted only after plan approval. A -reversible restart therefore needs an approved plan too. The tiers above govern -execution *within* that plan. Read-only telemetry goes through the read-only -instances, which are mounted by default, so it needs no plan. +elevated MCP server instances (F3), and those are mounted only after plan approval. +Read-only telemetry (metrics, logs, status) goes through the read-only instances, +which are mounted by default, so it needs no plan. -This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at -irreversible / high-blast-radius actions. +This is the sysadmin analog of the dev agent's plan gate. The dev agent has a second +gate after the work (the PR); the sysadmin agent does not, because a live change has +no review step before it takes effect. So its plan gate has to be exact. -Special cases fold into the tiers: -- **Browser actions against admin UIs default to confirm-first.** Changes to - **network gear (UniFi) are the most conservatively gated action in the system** — a - bad remote UI click can sever the path used to fix it. +**An ops plan is a list of literal steps, and approval covers exactly those steps.** +A write plan is submitted through `submit_plan` (A4) as structured data, not prose. +Each step names: +- the **tool and target**, e.g. the elevated Proxmox instance for `garrett-hall`, + node and VM ID; +- the **arguments**, in full. A config change carries the exact diff; +- an optional **precondition**, checked against the live system just before the + step runs; +- an optional **expected outcome**, checked just after, e.g. host reachable again, + config hash matches; +- a **retry allowance**, default one. + +One approval can cover a sequence with several high-consequence steps: "reboot the +host, change the config, reboot again" is one plan and one `approve`. The plan may +also carry prose explaining it, which is context for the operator, never scope. + +**The approval message is rendered by code, from the steps.** What the operator reads +is what will run; a model's description of a step is never the thing approved (see +A18). Steps that are irreversible, that touch a protected target, or that will take +down the agent's own host are flagged at the top of the message. + +**Execution is checked against the approved steps.** Every tool call passes through a +matcher before it reaches an MCP instance: +- **A read-only call is always allowed.** The agent can look around between steps. +- **An elevated call runs only if it exactly matches the next unfinished approved + step.** Arguments are normalized against the tool's JSON Schema before comparison + (types, key order), so a model's formatting cannot cause a false mismatch. +- **Anything else halts:** a call not in the plan, different arguments, steps out of + order, or a step run again after it completed. The agent moves to `AWAIT_INPUT`, + and the operator decides whether to continue, re-plan, or abandon. +- **A failed step may be retried** up to its allowance. A failed precondition or + expected-outcome check halts, without retry: the world does not match the plan, + and improvising around that is exactly what the gate exists to prevent. +- **A step that may have interrupted the agent** (a reboot of its own host) resumes + under A3's sequence rule. + +**Approvals expire.** An approved plan that has not started within the expiry window +must be approved again. An approval is a judgment about the world as it was when the +operator read the plan. The window is policy data (Part H). + +**Classification is small, conservative, and data.** The policy is a declared, +versioned file, like the egress allowlist (F2). It holds: +- **Instance privilege.** Read-only or elevated, per MCP instance (F3). This alone + decides whether a call needs an approved step. +- **Protected targets.** The NAS, the gateways, the hosts the agents run on, and + anything else whose loss is expensive. Protection changes how a step is + presented, not whether it is allowed. +- **Each elevated tool's target argument** (`vmid`, `node`, a device MAC), so the + matcher and the renderer can tell which target a call touches. An elevated tool + with no mapping is treated as touching a protected target. + +Defaults fail safe. Anything unclassified is treated as elevated. MCP tool annotations +(`readOnlyHint`, `destructiveHint`) are declared by the server itself, so they may +make a tool stricter, never looser. A model may propose entries for a new server's +tools offline, as a PR the operator reviews. Nothing classifies at run time. + +**Deferred, with the reason:** +- **Bounded matching** for reversible tools: allowing any call within an approved + target and action, not only the exact arguments. Added one tool at a time, where + exact matching causes real friction. It needs a per-tool reversibility + classification, which is the up-front cost exact matching avoids. +- **Standing approvals:** a plan template with a trigger, a rate limit and an + expiry, e.g. "restart Gitea on sustained 502, at most twice an hour, notify + after". The same matcher, run without a fresh `approve`. Added when a routine fix + has been approved by hand often enough to show its shape. + +Special cases: +- **Browser actions against admin UIs cannot be matched as steps,** since a click is + not a stable literal. Each one is confirm-first over transport, even within an + approved plan. Changes to **network gear (UniFi) are the most conservatively + gated action in the system**: a bad remote UI click can sever the path used to fix + it. - **Action-triggering tickets are operator-scoped** (C4). Auto-generated tickets - (e.g. from a monitoring alert) route through confirm-first, never drive unattended. + (e.g. from a monitoring alert) can only lead to a plan, which the operator + approves like any other. They never drive unattended. ### C3. Interfaces @@ -945,10 +1063,10 @@ Special cases fold into the tiers: The sysadmin analog of "dev agent driven by Gitea." - **Vikunja ticket = task of record** (durable, has status/queue/history): the external record of a work item (0.2). **Matrix = live conversation about it** - (questions, confirm-first prompts, progress). + (questions, halts, progress). - **Scheduled checks and alert intake are duties (0.2).** Tickets they generate are - duty findings, deduplicated against open tickets, and they route through - confirm-first (C2) like any auto-generated ticket. + duty findings, deduplicated against open tickets, and like any auto-generated + ticket they can only lead to a plan the operator approves (C2). - **Flow:** ticket filed → agent picks up → discuss/confirm over Matrix → update status as it progresses → close with result. - **Mechanics:** Vikunja REST API + webhooks (same two-direction, JSON-only pattern @@ -956,9 +1074,9 @@ The sysadmin analog of "dev agent driven by Gitea." installed version's webhooks fire on comment creation; if not, poll comments as B2 polls Gitea. - **Approvals:** Vikunja is an `ApprovalSource` (A4) for plan gates on its own - tickets, via `approve rev-N` comments. Live **confirm-first** prompts (C2) stay - Matrix-primary — polling latency makes a ticket comment a poor interactive - channel — with the outcome written back to the ticket as the record. + tickets, via `approve rev-N` comments. Live prompts (C2's halts and browser + confirmations) stay Matrix-primary — polling latency makes a ticket comment a poor + interactive channel — with the outcome written back to the ticket as the record. - **Trust:** a driving ticket is an instruction channel into an infrastructure- changing agent → action-triggering ticket creation is operator-scoped (C2). @@ -1153,7 +1271,7 @@ Acceptable only when consuming a third-party server whose surface we can't trim. 10. **Extract `nisse-core`**; build the **Sysadmin Agent** (`nisse-ops`): policy engine (C2), SSO + scoped browser (A8/A6), secrets (A7), Vikunja control (C4), knowledge store (A17/D) — world model first, operator procedures when the - C2 taxonomy they bind to exists — scheduled checks, alert intake. + C2 policy file they bind to exists — scheduled checks, alert intake. - With the monorepo layout (Part H, decided), extraction is a **package split** within one tree: move code behind a package boundary, no release machinery required. @@ -1198,8 +1316,19 @@ The rest resolve when reached, not now. path to network gear. It is a fallback for operations the UniFi API does not expose. The open question is whether that *fallback* access is standing or handed off per task. The lock-yourself-out risk still argues for per-task. -- **Policy action taxonomy (C2):** the concrete list of which operations are - allow vs. confirm. Best decided with the agent half-built in front of you. +- **Ops gating model (C2) — decided: approved step sequences, exact matching.** A + per-tool allow/confirm taxonomy was the original plan. It would have to be + written in advance for a tool catalog that will grow large, and every mistake in + it is either friction or a hole. Approving literal step sequences makes most of + it unnecessary: every elevated write has to be an approved step anyway, and + classification is left to decide only how an approval is *presented* (protected + targets, irreversible steps). What remains open is data, best filled in with the + agent half-built in front of you: the protected-target list, each elevated + tool's target argument, and the approval expiry window. +- **Parked: bounded matching and standing approvals (C2).** Both loosen exact + matching and reuse the same matcher. Add bounded matching one tool at a time, + where exact matching causes real friction. Add standing approvals when a routine + fix has been approved by hand often enough to show its shape. - **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are posted as issue comments on the project repo; the approval comment (author, timestamp, revision tag) is the durable off-box record, authenticated by Gitea's @@ -1218,13 +1347,14 @@ The rest resolve when reached, not now. - **Parked: more than one active work item (0.2).** Park-and-switch is the starting position. The pressure to relax it will come first from the Sysadmin Agent: an alert arriving mid-diagnosis, or a duty's scheduled run coming due while a work - item is active, has to wait for a safe point. Allowing concurrent items would - cost: + item is active, has to wait for a safe point. The agent's own switch (0.2) takes + the edge off: an alert waits for an item that is working, not for an operator + who is away. Allowing concurrent items would cost: - *Serialization in the state store and control loop.* Transitions for different items interleave, and the single orchestrator lock becomes per-item. - *A scheduler.* Something has to decide which item's turn runs, especially with one local inference host serving all of them. - - *Approval binding.* Two confirm-first prompts open at once make a bare `approve` + - *Approval binding.* Two plans or halts open at once make a bare `approve` ambiguous everywhere, so per-item rooms or strict reply threading become mandatory rather than convenient. - *Shared-scope conflicts.* Two active work items on one repo need separate @@ -1234,8 +1364,8 @@ The rest resolve when reached, not now. refusing to switch no longer keeps them single-item. - **Open: when a work item must have an external record (0.2).** Plan-gated dev work and ticket-driven ops work already require one. Whether every gated work item - needs one (for instance, an ops question that turns into a confirm-first change) - is best settled alongside the C2 taxonomy. + needs one (for instance, an ops question that turns into a change to make) + is best settled once C2's step sequences are in use. --- @@ -1260,10 +1390,10 @@ nisse-dev/ # app: sealed VM (Part B) browser → internal test network only nisse-ops/ # app: separate host, Tailscale (Part C) - policy engine, tiered confirmation, scheduled checks + policy engine (approved step sequences, C2), scheduled checks Vikunja ticket intake, Outline runbooks/records MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM - browser → admin UIs (API-first, UI fallback), confirm-first gating + browser → admin UIs (API-first, UI fallback), confirm-first per action infra/ # (future) Terraform + cloud-init (Part F) nisse-dev.tf | nisse-ops.tf | templates/