docs/Amendment-MCP-Deployment.md

@@ -1,157 +0,0 @@
-# Amendment — Agent Hosts, Egress, and MCP Server Deployment
-
-An amendment to `Architecture-Build-Spec.md`, recording decisions from a design session
-on MCP server deployment. It widens two decisions that were recorded as Dev Agent
-specific into rules for every agent, settles where MCP servers run and how they are
-split, and adjusts the secrets, permission, and image-delivery entries that those
-decisions touch.
-
-Each numbered section below names the spec locations it changes. After review, this
-amendment is folded into the spec and removed, as the Tiered Inference amendment was.
-
----
-
-## 1. Every agent gets its own VM
-
-**Changes:** new Part F1; B3 first bullet; C5.
-
-The spec records the dedicated VM as a Dev Agent decision, justified by
-Docker-in-Docker. That justification is specific to an agent that builds and runs
-containers. A future agent that does not build code would read it and correctly
-conclude it may run as a container. So the decision is promoted to a standing rule
-with its own rationale:
-
-- **Every agent gets its own VM.** No agent runs as a container on a shared host, and
-  no two agents share a VM.
-- **General rationale:**
-  - *Isolation boundary between agents.* The VM is the trust boundary (Part H,
-    monorepo decision). A compromised or misbehaving agent must not reach another
-    agent's state, credentials, or processes.
-  - *Per-agent credential attribution.* Everything an agent holds lives on its own
-    host and is issued to its own identity, so every upstream action traces back to
-    exactly one agent.
-- **Additional reason for the Dev Agent:** it builds and runs Docker containers, so it
-  needs its own real Docker daemon. Docker-in-a-container is either a socket mount,
-  which is close to host root, or dind, which brings nested-storage problems. This
-  reason is additive: it is not why the rule exists.
-
-## 2. Egress policy is per agent instance
-
-**Changes:** new Part F2; B3 "Sealed to start" bullet (becomes `nisse-dev`'s instance
-of the rule); C5.
-
-The Dev-Agent-scoped egress entry becomes a general rule:
-
-- **Every agent instance has its own default-deny allowlist.** Requirements differ by
-  agent type. No agent inherits another agent's policy.
-- **The allowlist is a declared artifact,** versioned alongside the agent's
-  definition. Each entry is justified by the component that needs it.
-- **The declaration point is fixed. The enforcement point is not.** Policy may be
-  enforced at the VM, at the Docker network, at the UniFi firewall, or at several of
-  these at once. Layering is deliberate: enforcement at the network layer survives
-  compromise of the agent's own host.
-- **The allowlist is the union** of what the agent reaches directly and what its MCP
-  servers (§3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi
-  endpoints at both sites, even though the agent never opens those connections
-  itself.
-- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A
-  rule written around RFC1918 denial does not cover it. Deny it explicitly.
-
-## 3. MCP servers: one instance per agent VM
-
-**Changes:** new Part F3.
-
-- **MCP servers run as containers on the owning agent's Docker host.** Images are
-  shared through the OCI registry. Processes are not shared between agents.
-- **Rationale:**
-  - *A shared process means a shared credential.* The upstream API's own audit log
-    then cannot attribute a call to a specific agent. Separate instances with
-    separate tokens give attribution at the API layer, instead of depending on our
-    own transcripts. This matters most for a read-only auditing agent, such as a
-    future `nisse-security`.
-  - *An agent should not host the tooling used to inspect it.*
-  - *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no
-    state.
-
-## 4. Per-target, per-privilege server instances
-
-**Changes:** Part F3; Part 0 (site identifiers); A7 (machine accounts); C2 (writes
-require an approved plan); C3.
-
-There is **one MCP server instance per (service, site, privilege level).**
-
-- **Site identifier is the domain stem:** `garrett-hall` (Arlington) and
-  `hartman-hut` (Cross Plains). The same string appears in the container name, the
-  Vaultwarden machine account, and the MCP tool prefix. Site belongs in the server's
-  identity, not in a `site` tool argument. A wrong-house action should require a tool
-  that isn't mounted, not just a wrong parameter value.
-- **Read-only and elevated access are separate instances** with separate upstream
-  credentials. The read-only instance is mounted by default. The elevated instance
-  is mounted only after plan approval. This puts the capability gate at the
-  credential layer, instead of a flag inside a third-party server.
-- **Per-task tool profiles** mount only the servers that a given task's scope
-  requires.
-
-Consequences for existing sections (settled with the operator while drafting this
-amendment):
-
-- **A7: a machine account per MCP server instance.** Each (service, site,
-  privilege) instance gets its own dedicated Vaultwarden machine account and
-  bootstrap token, alongside the agent's own. "One machine account per agent"
-  becomes "one per agent, plus one per MCP server instance." Each bootstrap token is
-  available only to its own container.
-- **C2: writes require an approved plan.** Every write credential sits behind the
-  elevated instance, so every write, including a reversible service action such as a
-  container restart, first needs an approved plan that mounts it. The C2 tiers then
-  govern execution *within* that plan: reversible actions run without further
-  confirmation, and high-consequence actions stay confirm-first. Read-only telemetry
-  stays free, through the read-only instance.
-
-## 5. Watchtower carve-out
-
-**Changes:** Part F4 (the existing Part F image-delivery entries).
-
-The existing pipeline (Gitea Actions → OCI registry → Watchtower, with restart gated
-on the idle flag) gets an exception. **Third-party MCP server images hold
-root-equivalent credentials to Proxmox and UniFi.** They are digest-pinned, excluded
-from Watchtower, and updated deliberately after review.
-
-## 6. Reframe the network-gear browser item
-
-**Changes:** Part H; C3.
-
-The open item "Sysadmin browser access to network gear: standing vs. per-task
-hand-off" is narrowed. With UniFi reachable over MCP, the browser becomes a fallback
-for API gaps, not the primary path. The open question is now whether the *fallback*
-browser access, used only for operations the UniFi API does not expose, is standing
-or handed off per task. The lockout risk still argues for per-task.
-
-## 7. Rejected alternative: routed meta-tools / lazy tool registration
-
-**Changes:** Part F3.
-
-Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large
-generated tool catalog. This is **rejected in favor of per-task tool profiles**:
-
-- *Two extra round-trips per call,* which is worse with a local model driving the
-  control loop.
-- *The model can no longer discover capabilities it wasn't already looking for.*
-  That is backwards for exploratory diagnostic work.
-- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON
-  Schema.
-- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what
-  makes a large static list affordable.
-
-Acceptable only when consuming a third-party server whose surface we can't trim.
-
-## 8. Parked: relocating stateful MCP servers
-
-**Changes:** Part H.
-
-Relocating stateful MCP servers off the owning agent's VM. The trigger is
-**statefulness** (a browser pool, a vector index, a warm cache), not sharing:
-duplicating a stateful server duplicates its state. Relocating has a cost. A shared
-server can't hold per-agent tokens, so per-agent identity has to move into the
-request, through caller auth plus per-caller upstream credentials. That loses the
-attribution property that makes instance-per-VM clean. Instance-per-agent-VM is the
-starting position. Migrate one server at a time, when a specific server justifies it.

docs/Architecture-Build-Spec.md

@@ -10,7 +10,10 @@ Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the
 document and no longer exists separately: its A-series components appear as A10–A16,
 its network and credential revisions in A5 and B3, and its revised build order as
 Part G. The inference-hardware question it raised has since been resolved in
-principle (Part H: a 4090-class host).
+principle (Part H: a 4090-class host). The MCP Deployment amendment has likewise been
+folded in: the every-agent-gets-a-VM rule, per-instance egress policy, and MCP server
+placement now form Part F1–F3; its consequences appear in Part 0 (site identifiers),
+A7, B3, C2, C3, C5, the Part F4 Watchtower carve-out, and Part H.
 
 ---
 
@@ -32,6 +35,11 @@ same string across all of those: the VM, the container, the Caddy host, the Matr
 account, and the Bitwarden machine account for a given agent all share one name, so
 that grepping the infrastructure for `nisse-ops` finds everything.
 
+**Sites** are identified by their domain stem: `garrett-hall` (Arlington) and
+`hartman-hut` (Cross Plains). As with agent names, one string is used everywhere a site
+appears. For an MCP server instance (F3), that means its container name, its
+Vaultwarden machine account, and its MCP tool prefix.
+
 The three artifacts live as three packages in **one repository, `nisse`** (decision
 recorded in Part H). Directory and package names carry the identity. If mirrored to
 GitHub, it goes under a personal account rather than an org — `github.com/nisse` is
@@ -261,9 +269,13 @@ Requirements:
 Requirements:
 - Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or
   `bw` CLI + API key). Not the personal vault.
-- Reads only from a **dedicated, minimally-scoped machine account** per agent.
+- Reads only from a **dedicated, minimally-scoped machine account** per agent, **plus
-- A single **bootstrap token** (the machine-account credential) is the one secret
+  one per MCP server instance** (F3). Each (service, site, privilege) instance holds
-  that lives on the box: tight file perms, never in image or repo.
+  its own upstream credential, so it gets its own machine account, and the agent's
+  account cannot read an instance's credential.
+- Each machine account has a single **bootstrap token**. These are the only secrets
+  that live on the box: tight file perms, never in image or repo. Each token is
+  available only to its own container.
 - (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the
   interface able to grow toward this; don't build it yet.)
 
@@ -746,13 +758,16 @@ PR_OPEN
 
 ### B3. Deployment & Isolation
 
-- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker
+- **Dedicated VM** (Proxmox), as for every agent (F1). The Dev Agent has an
-  containers, so it needs its own real Docker daemon. (Docker-in-a-container via
+  additional reason: it builds and runs Docker containers, so it needs its own real
-  socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the
+  Docker daemon. (Docker-in-a-container via socket-mount ≈ host root; via dind =
-  correct trust boundary for unattended arbitrary command execution.)
+  nested-storage pain. The VM boundary is the correct trust boundary for unattended
-- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no
+  arbitrary command execution.)
-  ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea
+- **Sealed to start:** `nisse-dev`'s egress allowlist (F2) permits LiteLLM, Gitea,
-  IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM.
+  and package registries, and there is **no ingress**. Egress denial must include
+  **RFC1918 except the specific LiteLLM/Gitea IPs**, and also the tailnet's
+  **`100.64.0.0/10`**, which is not RFC1918. Sealed from the internet must also mean
+  no reach to NAS/Proxmox/UDM.
   - **The LAN-resident inference host is not a second exception.** It sits behind
     LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how
     many models or vendors are in play.
@@ -789,9 +804,16 @@ deployments over the same library.
 
 Tiered policy, evaluated on every action:
 - **Read-only telemetry** (metrics, logs, status) — free.
-- **Reversible service actions** (restart a container) — allowed.
+- **Reversible service actions** (restart a container) — allowed within an approved
+  plan, without further confirmation.
 - **High-consequence actions** (storage, networking, auth) — **confirm-first** over
-  transport before executing.
+  transport before executing, even within an approved plan.
+
+**Every write requires an approved plan first.** Write credentials exist only in the
+elevated MCP server instances (F3), and those are mounted only after plan approval. A
+reversible restart therefore needs an approved plan too. The tiers above govern
+execution *within* that plan. Read-only telemetry goes through the read-only
+instances, which are mounted by default, so it needs no plan.
 
 This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
 irreversible / high-blast-radius actions.
@@ -805,9 +827,13 @@ Special cases fold into the tiers:
 
 ### C3. Interfaces
 
+- **MCP servers (F3):** Proxmox and UniFi, at both sites, through per-site,
+  per-privilege server instances on the agent's own Docker host. This is the primary
+  path to that infrastructure.
 - **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox,
-  TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps —
+  TrueNAS good; UniFi now reachable over MCP); use the browser only for genuinely
-  it's the least stable interface.
+  UI-only gaps. It's the least stable interface. For network gear, it is a fallback
+  for API gaps, not the primary path.
 - **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value
   credential; its blast radius equals its Authelia scope, so scope it minimally.
 - **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden.
@@ -835,8 +861,11 @@ The sysadmin analog of "dev agent driven by Gitea."
 
 ### C5. Deployment & Isolation
 
-- Longer-lived, **stateful VM** (not disposable).
+- Longer-lived, **stateful VM** (not disposable), dedicated to this agent (F1).
 - Has **Tailscale** + scoped credentials to reach infrastructure across both sites.
+- **Its MCP servers (F3) run on its own Docker host.** Its egress allowlist (F2)
+  therefore includes the Proxmox and UniFi endpoints at both sites, even though the
+  agent process never opens those connections itself.
 - Updated **in place** via the container pipeline, **restart gated on an idle flag**.
 
 ---
@@ -899,7 +928,89 @@ each agent is built — not standing up the service itself.
 
 ---
 
-## PART F — DEPLOYMENT MODEL (both agents)
+## PART F — DEPLOYMENT MODEL (all agents)
+
+These rules apply to every agent, current and future. Parts B3 and C5 record how each
+existing agent instantiates them.
+
+### F1. Agent hosts: every agent gets its own VM
+
+- **Every agent gets its own VM** (Proxmox). No agent runs as a container on a shared
+  host, and no two agents share a VM.
+- **Why, in general:**
+  - *Isolation boundary between agents.* The VM is the trust boundary (Part H). A
+    compromised or misbehaving agent must not reach another agent's state,
+    credentials, or processes.
+  - *Per-agent credential attribution.* Everything an agent holds lives on its own
+    host and is issued to its own identity, so every upstream action traces back to
+    exactly one agent.
+- **Agent-specific reasons add to this; they are not its foundation.** The Dev Agent
+  also needs a real Docker daemon, because it builds and runs containers (B3). An
+  agent that builds no code still gets a VM, for the two reasons above.
+
+### F2. Egress policy: per agent instance
+
+- **Every agent instance has its own default-deny allowlist.** Requirements differ by
+  agent type. No agent inherits another agent's policy.
+- **The allowlist is a declared artifact,** versioned alongside the agent's
+  definition. Each entry is justified by the component that needs it.
+- **The declaration point is fixed. The enforcement point is not.** Policy may be
+  enforced at the VM, at the Docker network, at the UniFi firewall, or at several of
+  these at once. Layering is deliberate: enforcement at the network layer survives
+  compromise of the agent's own host.
+- **The allowlist is the union** of what the agent reaches directly and what its MCP
+  servers (F3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi
+  endpoints at both sites, even though the agent never opens those connections
+  itself.
+- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A
+  rule written around RFC1918 denial does not cover it. Deny it explicitly.
+
+### F3. MCP servers
+
+**One instance per agent VM.** MCP servers run as containers on the owning agent's
+Docker host. Images are shared through the OCI registry. Processes are not shared
+between agents.
+
+- *A shared process means a shared credential.* The upstream API's own audit log
+  then cannot attribute a call to a specific agent. Separate instances with separate
+  tokens give attribution at the API layer, instead of depending on our own
+  transcripts. This matters most for a read-only auditing agent, such as a future
+  `nisse-security`.
+- *An agent should not host the tooling used to inspect it.*
+- *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no
+  state. (Stateful servers are the exception that could justify relocation. That
+  item is parked in Part H.)
+
+**One instance per (service, site, privilege level).**
+
+- **Site belongs in the server's identity,** using the site identifier (Part 0), not
+  in a `site` tool argument. A wrong-house action should require a tool that isn't
+  mounted, not just a wrong parameter value.
+- **Read-only and elevated access are separate instances** with separate upstream
+  credentials, each from its own machine account (A7). The read-only instance is
+  mounted by default. The elevated instance is mounted only after plan approval (C2).
+  This puts the capability gate at the credential layer, instead of a flag inside a
+  third-party server.
+- **Per-task tool profiles** mount only the servers that a given task's scope
+  requires.
+
+#### Rejected alternative: routed meta-tools / lazy tool registration
+
+Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large
+generated tool catalog. This is **rejected in favor of per-task tool profiles**:
+
+- *Two extra round-trips per call,* which is worse with a local model driving the
+  control loop.
+- *The model can no longer discover capabilities it wasn't already looking for.*
+  That is backwards for exploratory diagnostic work.
+- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON
+  Schema.
+- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what
+  makes a large static list affordable.
+
+Acceptable only when consuming a third-party server whose surface we can't trim.
+
+### F4. Containers, images, and restarts
 
 - **Containers + persistent volumes.** Stateless image; named volumes carry state
   across redeploys (and allow optional persistence between deployments).
@@ -907,6 +1018,9 @@ each agent is built — not standing up the service itself.
 - **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must
   never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the
   durability layers (A2/A3), not by restart timing.
+- **Carve-out: third-party MCP server images** (F3) are excluded from Watchtower.
+  They hold root-equivalent credentials to Proxmox and UniFi, so they are
+  digest-pinned and updated deliberately after review.
 - **Survive reboots:** VM + Docker service set to start on boot; `restart:
   unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval).
 - **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init,
@@ -976,8 +1090,11 @@ The rest resolve when reached, not now.
   are still moving. One repo can still build both agent images via path-filtered
   CI. Splitting later stays easy if a real need appears; merging later would not
   have been.
-- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The
+- **Sysadmin fallback browser access to network gear:** standing vs. per-task
-  lock-yourself-out risk argues for per-task.
+  hand-off. With UniFi reachable over MCP (F3), the browser is no longer the primary
+  path to network gear. It is a fallback for operations the UniFi API does not
+  expose. The open question is whether that *fallback* access is standing or handed
+  off per task. The lock-yourself-out risk still argues for per-task.
 - **Policy action taxonomy (C2):** the concrete list of which operations are
   allow vs. confirm. Best decided with the agent half-built in front of you.
 - **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
@@ -987,6 +1104,14 @@ The rest resolve when reached, not now.
   so boot-time reconciliation (A3) can re-derive gate state from Gitea even after
   state-volume loss — external truth wins (A2). Approval mechanics and revision
   binding are specified in A4.
+- **Parked: relocating stateful MCP servers off the owning agent's VM.** The trigger
+  is **statefulness** (a browser pool, a vector index, a warm cache), not sharing:
+  duplicating a stateful server duplicates its state. Relocating has a cost. A shared
+  server can't hold per-agent tokens, so per-agent identity has to move into the
+  request, through caller auth plus per-caller upstream credentials. That loses the
+  attribution property that makes instance-per-agent-VM (F3) clean.
+  Instance-per-agent-VM is the starting position. Migrate one server at a time, when
+  a specific server justifies it.
 
 ---
 
@@ -1013,6 +1138,7 @@ nisse-dev/               # app: sealed VM                        (Part B)
 nisse-ops/               # app: separate host, Tailscale         (Part C)
   policy engine, tiered confirmation, scheduled checks
   Vikunja ticket intake, Outline runbooks/records
+  MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM
   browser → admin UIs (API-first, UI fallback), confirm-first gating
 
 infra/                   # (future) Terraform + cloud-init       (Part F)