Gate ops writes on approved step sequences #39

Merged
dongar merged 5 commits from ops-gating into main 2026-09-23 22:26:13 +00:00
Showing only changes of commit b8df5d2066 - Show all commits
+213 -83
View File
@@ -159,6 +159,17 @@ come back to something. Components throughout this document refer to them.
arrive is a safe point, with one exception: switching is refused while intents
are unresolved (A2). That keeps every pending intent owned by the active work
item, so boot reconciliation (A3) stays single-item.
- **The operator can always switch; the agent may switch on its own in one case.**
When the active work item is waiting on the operator (an open gate, a pending
question, or a halt in `AWAIT_INPUT`) and another work item is ready, the agent
switches to it, oldest first, and posts a notice naming both items. An alert
must not sit for hours behind an unanswered prompt. The agent never switches
away from an item that is working. The item it left stays waiting; switching
answers nothing on its behalf.
- **Parking gives up authority.** Parking a work item unmounts any elevated MCP
instances (F3) its approved plan mounted. They are remounted when it becomes
active again, if its approval has not expired (C2). An approved plan's
credentials never serve another item.
- **Nothing is lost while parked.** Intake for a parked work item, such as comments
on its Gitea issue or PR, is read from a durable per-thread cursor when it becomes
active again. It is not dropped, and it is not replayed from the beginning.
@@ -244,10 +255,18 @@ Every startup (clean or post-crash), before accepting operator commands:
(tree diff for edits, Gitea query for pushes), or on any branch/log mismatch. The
judgment is an observable check, never model self-assessment — the same reasoning
as A12.
6. Any other interruption → move to **`AWAIT_INPUT`** and post to transport
6. **Interrupted mid-sequence (C2) — resume only on evidence.** An approved step
sequence may interrupt itself: rebooting the host the agent runs on is a
legitimate step. On restart, **continue** the sequence only when the intent log
names exactly which step was in flight and that step's expected outcome checks
out against the live system, and the approval has not expired. Anything else
(no recorded outcome check, a failed check, or more than one step
possibly in flight) gates to `AWAIT_INPUT`. The step counts as completed only
by that check, never by assuming it ran.
7. Any other interruption → move to **`AWAIT_INPUT`** and post to transport
(e.g. "restarted mid-milestone-2, branch at commit abc123, continue or re-plan?").
Never blind-resume.
7. Otherwise resume the last committed state.
8. Otherwise resume the last committed state.
### A4. Control Loop, State Machine, and Gating Primitives
@@ -289,11 +308,20 @@ Requirements:
- **Approvals bind to a work item, too (0.2).** A bare `approve` binds to the
*active* work item's latest revision, under the unambiguity rule above. An
approval arriving on a *parked* item's external record (a Gitea or Vikunja
comment) is delivered when that item next becomes active. It is never applied to
the active item.
comment) is delivered when that item next becomes active. So is a Matrix reply to
a parked item's plan or prompt: the reply relation binds it to that item. Neither
is ever applied to the active item.
- **Autonomy inside an approved unit of work:** all tool/command use auto-approved
(Claude Agent SDK `bypassPermissions` or an allow-all `canUseTool`). Escalation to
the human happens *only* via `ask_human`.
the human happens *only* via `ask_human`. For the Sysadmin Agent, "approved"
means the step list of C2. Only calls that match an approved step are
auto-approved.
- **Kill switch.** The operator command `stop` works in every state, for every
agent. It ends the current turn or delegation at the next tool boundary, unmounts
elevated MCP instances (F3), and moves to `AWAIT_INPUT`. The unfinished delegation
or step sequence is then handled like any other halt: continue, re-plan, or
abandon, on the operator's word. Unattended work needs a way to end it that does
not depend on the model cooperating.
- The loop must survive restart mid-work (relies on A2/A3).
### A5. LLM Access *(tiered)*
@@ -642,33 +670,24 @@ Part D describes the service; this defines the contract.
desynchronise from the project.
**Agent-owned knowledge is always a cache, never authority.** An agent may rely on
it to know *what exists and where to look*; it must confirm a specific fact
against the live system *before acting on it*. Hard requirement for anything in
C2's confirm-first tier; advisory for read-only and reversible actions.
it to know *what exists and where to look*. It must not rely on it for the facts an
action depends on. A prompt cannot enforce that: the model cannot be prevented from
*believing* a fact it has read. The enforcement is elsewhere, and it is structural:
- **Step preconditions (C2) carry the weight.** Every elevated write is an approved
step, and a step that depends on a fact states it as a precondition, checked
against the live system immediately before the step runs. A stale world model
can produce a wrong plan, but it cannot turn that plan into an action on a
system that no longer matches it.
- **Provenance at the human gate.** When a plan is drafted from recalled facts, the
approval message says so: "target recalled from the world model, observed
2026-08-02" versus "confirmed by `zfs list` two minutes ago". The operator is
the authority there; this makes the evidence quality visible.
- **Audit for the rest.** Retrievals are logged (A10), so a wrong fact can be traced
to the actions that followed it.
**Enforcement is structural, because a prompt is not enforcement.** "Treat this as
a cache" is exactly the advisory guarantee A4 and A12 reject, and the model cannot
be prevented from *believing* a fact it has read. The enforceable goal is
narrower: **an unverified belief must not become an irreversible action.**
1. **The store decides what it serves.** Facts carry provenance and an observed-at
timestamp; retrieval marks or withholds anything past a staleness bound for
high-consequence classes.
2. **Verification as a precondition, checked against the log.** For confirm-first
actions, refuse unless a verifying observation exists *in this task's canonical
log*, recorded after the task began, naming the action's target. Inspectable
without asking the model anything — the same move as A12's observable triggers.
Approximate by nature, so build it for the confirm-first tier only.
3. **Provenance at the human gate** — the primary reliance, because C2 already
stops these actions for confirmation. The prompt states evidence quality:
"target recalled from the world model, observed 2026-08-02, not verified this
session" versus "confirmed by `zfs list` two minutes ago". The operator is
already the authority there; this makes the risk legible rather than invisible.
4. **Audit, not prevention, for the cheap tiers.** Log which facts were retrieved
and which actions followed, so a wrong fact is traceable afterwards.
Residual risk, stated plainly: a model can retype a fact from context with no
retrieval event, and no mechanism here catches that. That is why the world model
stays out of the authority path entirely.
**Dropped:** withholding facts past a staleness bound at retrieval, and refusing
confirm-first actions without a verifying observation in the log. Both were
approximations of what step preconditions (C2) now check directly.
**Never an instruction channel, and never an approval channel.**
- **Task instructions never come from the knowledge store.** Work is driven by
@@ -699,15 +718,14 @@ a model rewriting a precise fact loses precision invisibly.
log** — the text, plus document id and revision for provenance — never referenced
by handle alone: A10 forbids a provider handle being the only reference to content,
and these documents are mutable. Snapshot once per task at first use, not per turn.
**A3 consequence:** a task resumed across a restart carries the old snapshot, so
boot reconciliation re-fetches snapshotted knowledge and flags divergence rather
than silently continuing on a stale copy.
A task resumed across a restart carries the old snapshot; step preconditions, not
a re-fetch, are what stop it acting on a stale copy.
**Retrieval is structural first, discretionary second** — the A12/A14 pattern.
- Procedures are **bound to C2 action classes**: the policy taxonomy and the
procedure index are the same list. Binding is deterministic and auditable
("which procedure governed this action?" has an exact answer), and costs nothing
extra because the taxonomy must exist anyway.
- Procedures are **bound to entries in the C2 policy file** (a tool, or a protected
target), and fetched when a plan that uses that entry is drafted. Binding is
deterministic and auditable ("which procedure governed this action?" has an
exact answer), and costs little because the policy file must exist anyway.
- A model-invoked **search** exists *in addition* for discovery, never as a
replacement for the bound retrieval — a procedure the model may skip does not
constrain it.
@@ -715,10 +733,10 @@ than silently continuing on a stale copy.
**Outage posture is asymmetric.** Unlike B2's poller, "never fails a task" does not
generalise here.
- **Write failure never blocks:** retry, or record locally and reconcile.
- **Read failure for a procedure bound to a confirm-first action gates.**
Proceeding is not a delayed reaction; it is acting unconstrained on precisely the
operation someone wrote a procedure for.
- Read failure for read-only or reversible actions proceeds, logging the gap.
- **Read failure for a bound procedure gates the plan.** Planning without it is not
a delayed reaction; it is acting unconstrained on precisely the operation someone
wrote a procedure for.
- Read failure for anything else proceeds, logging the gap.
**Accepted single point of failure.** Once the agent is the sole author and Outline
the only home, losing Outline loses the world model. No local mirror: infrastructure
@@ -726,7 +744,8 @@ facts are re-derivable by observation, which is the same premise the cache rule
rests on.
**Deferred:** the operator-writable procedures collection. The interface is defined
here; the content and the C2 binding list arrive with the policy taxonomy (Part H).
here; the content arrives when there are procedures worth writing, and binds to the
C2 policy file.
**Concrete mapping onto Outline.** Observed in the running instance: a **collection**
carries per-user permissions with three tiers — visible-to (read), read-write, and
@@ -764,9 +783,9 @@ manager. Permissions are collection-scoped, not per-document.
Outline can expose search, read, create and edit over MCP (remote, Streamable HTTP,
enabled workspace-wide). **Not adopted for the retrieval path**, because A17's
enforcement depends on this library sitting *between* the model and the store: staleness
marking, snapshotting into the canonical log, binding procedures to action classes, and
gating a failed confirm-first read all happen to a retrieval before the model sees it.
contract depends on this library sitting *between* the model and the store:
snapshotting into the canonical log, binding procedures to C2 policy entries, and
gating a failed bound read all happen to a retrieval before the model sees it.
MCP's purpose is the opposite — direct model access — and a procedure the model may
decline to fetch does not constrain it (A12).
@@ -778,6 +797,43 @@ membership, and that the workspace-level toggle does not force broader exposure
intended.
### A18. Prompt Injection
**Assume any model that reads untrusted content may be compromised.** There is no
robust way to tell a model "this is data, not instruction". Instructions and data
reach it as one token stream, and every marker meant to separate them is text that
the data can imitate or argue against. Role separation, delimiters and detection
classifiers lower the success rate of injection. None of them bring it to zero, and
an attacker gets to try repeatedly. So the design does not depend on the model
telling them apart. It limits what a compromised model can do.
Untrusted content, for these agents: alert payloads, log lines, ticket and comment
text, web pages the browser reads, repository contents, and downloaded packages.
Requirements:
- **Gates hold against a compromised model.** Nothing a model says can approve a
plan (A4), widen what a plan authorizes (C2), or reach a credential that was not
mounted (F3). Every one of those rules is enforced in code, outside the model.
- **Approval messages are rendered by code from what will actually run** (C2),
never from a model's description of it. Otherwise a compromised model can write
"restart Gitea" above a step that destroys a dataset.
- **Instruction channels are operator-scoped.** Only allowlisted identities can
drive work: operator commands (A1), PR review comments (B2), action-triggering
tickets (C4). Content from anyone else is data.
- **Parse intake into structured fields in code** wherever the source allows it,
e.g. alert name, severity, host, from Alertmanager's JSON. Less free text then
reaches the model at all, and what reaches it is labelled by where it came from.
- **Check each agent against the lethal trifecta:** access to private data, exposure
to untrusted content, and a channel to send data out. Damage needs all three.
- `nisse-dev` reads untrusted content and holds little private data: its Gitea
token and the repos it works on. Its only ways out are Gitea and package
registries (B3). The registries are an accepted, narrow residual channel.
Routing them through a local mirror would close it, if that ever becomes worth
doing.
- `nisse-ops` has all three: it reads alerts and logs, holds infrastructure
credentials, and has broad egress. It relies on the gates above, not on
removing a leg.
---
## PART B — THE DEV AGENT / `nisse-dev` (build first)
@@ -813,8 +869,8 @@ PR_OPEN
- **One active project at a time** (park-and-switch, 0.2). Operator selects it;
other projects stay parked at their last position and resume where they left off.
- **Operator commands** (over transport): `use <project>`, `approve`,
`reject <feedback>`, `start milestone <n>`, `status`, plus free text during
planning (forwarded to the agent as answers).
`reject <feedback>`, `start milestone <n>`, `status`, `stop` (A4), plus free
text during planning (forwarded to the agent as answers).
- **Plan gating** per A4: architecture plan and each milestone plan require sign-off
before the next phase. Enforced structurally. Plans and approvals are externalized
to Gitea issue comments (Part H, decided); Matrix and Gitea are both approval
@@ -891,7 +947,7 @@ tickets; gated on consequences. Built after the Dev Agent works.
| Axis | Dev Agent | Sysadmin Agent |
|---|---|---|
| Rollback | revert snapshot / discard branch | none for `zfs destroy` or a firewall lockout |
| Trust model | allow-all in sandbox | tiered **allow / deny / confirm** policy |
| Trust model | allow-all in sandbox | **approved step sequences**, checked on every call |
| Work shape | long milestones → a PR | short reactive tasks + scheduled checks → live changes |
| Driver | Gitea projects | **Vikunja tickets** |
| Placement | sealed, own VLAN, no Tailscale | Tailscale, credentials, broad reach |
@@ -901,28 +957,90 @@ deployments over the same library.
### C2. Permission Policy Engine *(the defining addition)*
Tiered policy, evaluated on every action:
- **Read-only telemetry** (metrics, logs, status) — free.
- **Reversible service actions** (restart a container) — allowed within an approved
plan, without further confirmation.
- **High-consequence actions** (storage, networking, auth) — **confirm-first** over
transport before executing, even within an approved plan.
**Every write requires an approved plan first.** Write credentials exist only in the
elevated MCP server instances (F3), and those are mounted only after plan approval. A
reversible restart therefore needs an approved plan too. The tiers above govern
execution *within* that plan. Read-only telemetry goes through the read-only
instances, which are mounted by default, so it needs no plan.
elevated MCP server instances (F3), and those are mounted only after plan approval.
Read-only telemetry (metrics, logs, status) goes through the read-only instances,
which are mounted by default, so it needs no plan.
This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
irreversible / high-blast-radius actions.
This is the sysadmin analog of the dev agent's plan gate. The dev agent has a second
gate after the work (the PR); the sysadmin agent does not, because a live change has
no review step before it takes effect. So its plan gate has to be exact.
Special cases fold into the tiers:
- **Browser actions against admin UIs default to confirm-first.** Changes to
**network gear (UniFi) are the most conservatively gated action in the system** — a
bad remote UI click can sever the path used to fix it.
**An ops plan is a list of literal steps, and approval covers exactly those steps.**
A write plan is submitted through `submit_plan` (A4) as structured data, not prose.
Each step names:
- the **tool and target**, e.g. the elevated Proxmox instance for `garrett-hall`,
node and VM ID;
- the **arguments**, in full. A config change carries the exact diff;
- an optional **precondition**, checked against the live system just before the
step runs;
- an optional **expected outcome**, checked just after, e.g. host reachable again,
config hash matches;
- a **retry allowance**, default one.
One approval can cover a sequence with several high-consequence steps: "reboot the
host, change the config, reboot again" is one plan and one `approve`. The plan may
also carry prose explaining it, which is context for the operator, never scope.
**The approval message is rendered by code, from the steps.** What the operator reads
is what will run; a model's description of a step is never the thing approved (see
A18). Steps that are irreversible, that touch a protected target, or that will take
down the agent's own host are flagged at the top of the message.
**Execution is checked against the approved steps.** Every tool call passes through a
matcher before it reaches an MCP instance:
- **A read-only call is always allowed.** The agent can look around between steps.
- **An elevated call runs only if it exactly matches the next unfinished approved
step.** Arguments are normalized against the tool's JSON Schema before comparison
(types, key order), so a model's formatting cannot cause a false mismatch.
- **Anything else halts:** a call not in the plan, different arguments, steps out of
order, or a step run again after it completed. The agent moves to `AWAIT_INPUT`,
and the operator decides whether to continue, re-plan, or abandon.
- **A failed step may be retried** up to its allowance. A failed precondition or
expected-outcome check halts, without retry: the world does not match the plan,
and improvising around that is exactly what the gate exists to prevent.
- **A step that may have interrupted the agent** (a reboot of its own host) resumes
under A3's sequence rule.
**Approvals expire.** An approved plan that has not started within the expiry window
must be approved again. An approval is a judgment about the world as it was when the
operator read the plan. The window is policy data (Part H).
**Classification is small, conservative, and data.** The policy is a declared,
versioned file, like the egress allowlist (F2). It holds:
- **Instance privilege.** Read-only or elevated, per MCP instance (F3). This alone
decides whether a call needs an approved step.
- **Protected targets.** The NAS, the gateways, the hosts the agents run on, and
anything else whose loss is expensive. Protection changes how a step is
presented, not whether it is allowed.
- **Each elevated tool's target argument** (`vmid`, `node`, a device MAC), so the
matcher and the renderer can tell which target a call touches. An elevated tool
with no mapping is treated as touching a protected target.
Defaults fail safe. Anything unclassified is treated as elevated. MCP tool annotations
(`readOnlyHint`, `destructiveHint`) are declared by the server itself, so they may
make a tool stricter, never looser. A model may propose entries for a new server's
tools offline, as a PR the operator reviews. Nothing classifies at run time.
**Deferred, with the reason:**
- **Bounded matching** for reversible tools: allowing any call within an approved
target and action, not only the exact arguments. Added one tool at a time, where
exact matching causes real friction. It needs a per-tool reversibility
classification, which is the up-front cost exact matching avoids.
- **Standing approvals:** a plan template with a trigger, a rate limit and an
expiry, e.g. "restart Gitea on sustained 502, at most twice an hour, notify
after". The same matcher, run without a fresh `approve`. Added when a routine fix
has been approved by hand often enough to show its shape.
Special cases:
- **Browser actions against admin UIs cannot be matched as steps,** since a click is
not a stable literal. Each one is confirm-first over transport, even within an
approved plan. Changes to **network gear (UniFi) are the most conservatively
gated action in the system**: a bad remote UI click can sever the path used to fix
it.
- **Action-triggering tickets are operator-scoped** (C4). Auto-generated tickets
(e.g. from a monitoring alert) route through confirm-first, never drive unattended.
(e.g. from a monitoring alert) can only lead to a plan, which the operator
approves like any other. They never drive unattended.
### C3. Interfaces
@@ -945,10 +1063,10 @@ Special cases fold into the tiers:
The sysadmin analog of "dev agent driven by Gitea."
- **Vikunja ticket = task of record** (durable, has status/queue/history): the
external record of a work item (0.2). **Matrix = live conversation about it**
(questions, confirm-first prompts, progress).
(questions, halts, progress).
- **Scheduled checks and alert intake are duties (0.2).** Tickets they generate are
duty findings, deduplicated against open tickets, and they route through
confirm-first (C2) like any auto-generated ticket.
duty findings, deduplicated against open tickets, and like any auto-generated
ticket they can only lead to a plan the operator approves (C2).
- **Flow:** ticket filed → agent picks up → discuss/confirm over Matrix → update
status as it progresses → close with result.
- **Mechanics:** Vikunja REST API + webhooks (same two-direction, JSON-only pattern
@@ -956,9 +1074,9 @@ The sysadmin analog of "dev agent driven by Gitea."
installed version's webhooks fire on comment creation; if not, poll comments as
B2 polls Gitea.
- **Approvals:** Vikunja is an `ApprovalSource` (A4) for plan gates on its own
tickets, via `approve rev-N` comments. Live **confirm-first** prompts (C2) stay
Matrix-primary — polling latency makes a ticket comment a poor interactive
channel — with the outcome written back to the ticket as the record.
tickets, via `approve rev-N` comments. Live prompts (C2's halts and browser
confirmations) stay Matrix-primary — polling latency makes a ticket comment a poor
interactive channel — with the outcome written back to the ticket as the record.
- **Trust:** a driving ticket is an instruction channel into an infrastructure-
changing agent → action-triggering ticket creation is operator-scoped (C2).
@@ -1153,7 +1271,7 @@ Acceptable only when consuming a third-party server whose surface we can't trim.
10. **Extract `nisse-core`**; build the **Sysadmin Agent** (`nisse-ops`): policy
engine (C2), SSO + scoped browser (A8/A6), secrets (A7), Vikunja control (C4),
knowledge store (A17/D) — world model first, operator procedures when the
C2 taxonomy they bind to exists — scheduled checks, alert intake.
C2 policy file they bind to exists — scheduled checks, alert intake.
- With the monorepo layout (Part H, decided), extraction is a **package split**
within one tree: move code behind a package boundary, no release machinery
required.
@@ -1198,8 +1316,19 @@ The rest resolve when reached, not now.
path to network gear. It is a fallback for operations the UniFi API does not
expose. The open question is whether that *fallback* access is standing or handed
off per task. The lock-yourself-out risk still argues for per-task.
- **Policy action taxonomy (C2):** the concrete list of which operations are
allow vs. confirm. Best decided with the agent half-built in front of you.
- **Ops gating model (C2) — decided: approved step sequences, exact matching.** A
per-tool allow/confirm taxonomy was the original plan. It would have to be
written in advance for a tool catalog that will grow large, and every mistake in
it is either friction or a hole. Approving literal step sequences makes most of
it unnecessary: every elevated write has to be an approved step anyway, and
classification is left to decide only how an approval is *presented* (protected
targets, irreversible steps). What remains open is data, best filled in with the
agent half-built in front of you: the protected-target list, each elevated
tool's target argument, and the approval expiry window.
- **Parked: bounded matching and standing approvals (C2).** Both loosen exact
matching and reuse the same matcher. Add bounded matching one tool at a time,
where exact matching causes real friction. Add standing approvals when a routine
fix has been approved by hand often enough to show its shape.
- **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
posted as issue comments on the project repo; the approval comment (author,
timestamp, revision tag) is the durable off-box record, authenticated by Gitea's
@@ -1218,13 +1347,14 @@ The rest resolve when reached, not now.
- **Parked: more than one active work item (0.2).** Park-and-switch is the starting
position. The pressure to relax it will come first from the Sysadmin Agent: an
alert arriving mid-diagnosis, or a duty's scheduled run coming due while a work
item is active, has to wait for a safe point. Allowing concurrent items would
cost:
item is active, has to wait for a safe point. The agent's own switch (0.2) takes
the edge off: an alert waits for an item that is working, not for an operator
who is away. Allowing concurrent items would cost:
- *Serialization in the state store and control loop.* Transitions for different
items interleave, and the single orchestrator lock becomes per-item.
- *A scheduler.* Something has to decide which item's turn runs, especially with
one local inference host serving all of them.
- *Approval binding.* Two confirm-first prompts open at once make a bare `approve`
- *Approval binding.* Two plans or halts open at once make a bare `approve`
ambiguous everywhere, so per-item rooms or strict reply threading become
mandatory rather than convenient.
- *Shared-scope conflicts.* Two active work items on one repo need separate
@@ -1234,8 +1364,8 @@ The rest resolve when reached, not now.
refusing to switch no longer keeps them single-item.
- **Open: when a work item must have an external record (0.2).** Plan-gated dev work
and ticket-driven ops work already require one. Whether every gated work item
needs one (for instance, an ops question that turns into a confirm-first change)
is best settled alongside the C2 taxonomy.
needs one (for instance, an ops question that turns into a change to make)
is best settled once C2's step sequences are in use.
---
@@ -1260,10 +1390,10 @@ nisse-dev/ # app: sealed VM (Part B)
browser → internal test network only
nisse-ops/ # app: separate host, Tailscale (Part C)
policy engine, tiered confirmation, scheduled checks
policy engine (approved step sequences, C2), scheduled checks
Vikunja ticket intake, Outline runbooks/records
MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM
browser → admin UIs (API-first, UI fallback), confirm-first gating
browser → admin UIs (API-first, UI fallback), confirm-first per action
infra/ # (future) Terraform + cloud-init (Part F)
nisse-dev.tf | nisse-ops.tf | templates/