Merge pull request 'Record the MCP server deployment decisions in the spec' (#36) from mcp-deployment into main

Reviewed-on: dongar/nisse#36
Reviewed-by: Don Garrett <dgarrett@acm.org>

docs/Architecture-Build-Spec.md

@@ -10,7 +10,10 @@ Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the
 document and no longer exists separately: its A-series components appear as A10–A16,
 its network and credential revisions in A5 and B3, and its revised build order as
 Part G. The inference-hardware question it raised has since been resolved in
-principle (Part H: a 4090-class host).
+principle (Part H: a 4090-class host). The MCP Deployment amendment has likewise been
+folded in: the every-agent-gets-a-VM rule, per-instance egress policy, and MCP server
+placement now form Part F1–F3; its consequences appear in Part 0 (site identifiers),
+A7, B3, C2, C3, C5, the Part F4 Watchtower carve-out, and Part H.
 
 ---
 
@@ -32,6 +35,12 @@ same string across all of those: the VM, the container, the Caddy host, the Matr
 account, and the Bitwarden machine account for a given agent all share one name, so
 that grepping the infrastructure for `nisse-ops` finds everything.
 
+**Sites** are identified by their domain stem: `garrett-hall` (Arlington, hosts under
+`garrett-hall.com`) and `hartman-hut` (Cross Plains, hosts under `hartman-hut.com`).
+As with agent names, one string is used everywhere a site appears. For an MCP server
+instance (F3), that means its container name, its Vaultwarden machine account, and
+its MCP tool prefix.
+
 The three artifacts live as three packages in **one repository, `nisse`** (decision
 recorded in Part H). Directory and package names carry the identity. If mirrored to
 GitHub, it goes under a personal account rather than an org — `github.com/nisse` is
@@ -261,9 +270,13 @@ Requirements:
 Requirements:
 - Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or
   `bw` CLI + API key). Not the personal vault.
-- Reads only from a **dedicated, minimally-scoped machine account** per agent.
+- Reads only from a **dedicated, minimally-scoped machine account** per agent, **plus
-- A single **bootstrap token** (the machine-account credential) is the one secret
+  one per MCP server instance** (F3). Each (service, site, privilege) instance holds
-  that lives on the box: tight file perms, never in image or repo.
+  its own upstream credential, so it gets its own machine account, and the agent's
+  account cannot read an instance's credential.
+- Each machine account has a single **bootstrap token**. These are the only secrets
+  that live on the box: tight file perms, never in image or repo. Each token is
+  available only to its own container.
 - (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the
   interface able to grow toward this; don't build it yet.)
 
@@ -746,13 +759,16 @@ PR_OPEN
 
 ### B3. Deployment & Isolation
 
-- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker
+- **Dedicated VM** (Proxmox), as for every agent (F1). The Dev Agent has an
-  containers, so it needs its own real Docker daemon. (Docker-in-a-container via
+  additional reason: it builds and runs Docker containers, so it needs its own real
-  socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the
+  Docker daemon. (Docker-in-a-container via socket-mount ≈ host root; via dind =
-  correct trust boundary for unattended arbitrary command execution.)
+  nested-storage pain. The VM boundary is the correct trust boundary for unattended
-- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no
+  arbitrary command execution.)
-  ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea
+- **Sealed to start:** `nisse-dev`'s egress allowlist (F2) permits LiteLLM, Gitea,
-  IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM.
+  and package registries, and there is **no ingress**. Egress denial must include
+  **RFC1918 except the specific LiteLLM/Gitea IPs**, and also the tailnet's
+  **`100.64.0.0/10`**, which is not RFC1918. Sealed from the internet must also mean
+  no reach to NAS/Proxmox/UDM.
   - **The LAN-resident inference host is not a second exception.** It sits behind
     LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how
     many models or vendors are in play.
@@ -789,9 +805,16 @@ deployments over the same library.
 
 Tiered policy, evaluated on every action:
 - **Read-only telemetry** (metrics, logs, status) — free.
-- **Reversible service actions** (restart a container) — allowed.
+- **Reversible service actions** (restart a container) — allowed within an approved
+  plan, without further confirmation.
 - **High-consequence actions** (storage, networking, auth) — **confirm-first** over
-  transport before executing.
+  transport before executing, even within an approved plan.
+
+**Every write requires an approved plan first.** Write credentials exist only in the
+elevated MCP server instances (F3), and those are mounted only after plan approval. A
+reversible restart therefore needs an approved plan too. The tiers above govern
+execution *within* that plan. Read-only telemetry goes through the read-only
+instances, which are mounted by default, so it needs no plan.
 
 This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
 irreversible / high-blast-radius actions.
@@ -805,9 +828,13 @@ Special cases fold into the tiers:
 
 ### C3. Interfaces
 
+- **MCP servers (F3):** Proxmox and UniFi, at both sites, through per-site,
+  per-privilege server instances on the agent's own Docker host. This is the primary
+  path to that infrastructure.
 - **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox,
-  TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps —
+  TrueNAS good; UniFi now reachable over MCP); use the browser only for genuinely
-  it's the least stable interface.
+  UI-only gaps. It's the least stable interface. For network gear, it is a fallback
+  for API gaps, not the primary path.
 - **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value
   credential; its blast radius equals its Authelia scope, so scope it minimally.
 - **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden.
@@ -835,8 +862,11 @@ The sysadmin analog of "dev agent driven by Gitea."
 
 ### C5. Deployment & Isolation
 
-- Longer-lived, **stateful VM** (not disposable).
+- Longer-lived, **stateful VM** (not disposable), dedicated to this agent (F1).
 - Has **Tailscale** + scoped credentials to reach infrastructure across both sites.
+- **Its MCP servers (F3) run on its own Docker host.** Its egress allowlist (F2)
+  therefore includes the Proxmox and UniFi endpoints at both sites, even though the
+  agent process never opens those connections itself.
 - Updated **in place** via the container pipeline, **restart gated on an idle flag**.
 
 ---
@@ -899,7 +929,89 @@ each agent is built — not standing up the service itself.
 
 ---
 
-## PART F — DEPLOYMENT MODEL (both agents)
+## PART F — DEPLOYMENT MODEL (all agents)
+
+These rules apply to every agent, current and future. Parts B3 and C5 record how each
+existing agent instantiates them.
+
+### F1. Agent hosts: every agent gets its own VM
+
+- **Every agent gets its own VM** (Proxmox). No agent runs as a container on a shared
+  host, and no two agents share a VM.
+- **Why, in general:**
+  - *Isolation boundary between agents.* The VM is the trust boundary (Part H). A
+    compromised or misbehaving agent must not reach another agent's state,
+    credentials, or processes.
+  - *Per-agent credential attribution.* Everything an agent holds lives on its own
+    host and is issued to its own identity, so every upstream action traces back to
+    exactly one agent.
+- **Agent-specific reasons add to this; they are not its foundation.** The Dev Agent
+  also needs a real Docker daemon, because it builds and runs containers (B3). An
+  agent that builds no code still gets a VM, for the two reasons above.
+
+### F2. Egress policy: per agent instance
+
+- **Every agent instance has its own default-deny allowlist.** Requirements differ by
+  agent type. No agent inherits another agent's policy.
+- **The allowlist is a declared artifact,** versioned alongside the agent's
+  definition. Each entry is justified by the component that needs it.
+- **The declaration point is fixed. The enforcement point is not.** Policy may be
+  enforced at the VM, at the Docker network, at the UniFi firewall, or at several of
+  these at once. Layering is deliberate: enforcement at the network layer survives
+  compromise of the agent's own host.
+- **The allowlist is the union** of what the agent reaches directly and what its MCP
+  servers (F3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi
+  endpoints at both sites, even though the agent never opens those connections
+  itself.
+- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A
+  rule written around RFC1918 denial does not cover it. Deny it explicitly.
+
+### F3. MCP servers
+
+**One instance per agent VM.** MCP servers run as containers on the owning agent's
+Docker host. Images are shared through the OCI registry. Processes are not shared
+between agents.
+
+- *A shared process means a shared credential.* The upstream API's own audit log
+  then cannot attribute a call to a specific agent. Separate instances with separate
+  tokens give attribution at the API layer, instead of depending on our own
+  transcripts. This matters most for a read-only auditing agent, such as a future
+  `nisse-security`.
+- *An agent should not host the tooling used to inspect it.*
+- *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no
+  state. (Stateful servers are the exception that could justify relocation. That
+  item is parked in Part H.)
+
+**One instance per (service, site, privilege level).**
+
+- **Site belongs in the server's identity,** using the site identifier (Part 0), not
+  in a `site` tool argument. A wrong-house action should require a tool that isn't
+  mounted, not just a wrong parameter value.
+- **Read-only and elevated access are separate instances** with separate upstream
+  credentials, each from its own machine account (A7). The read-only instance is
+  mounted by default. The elevated instance is mounted only after plan approval (C2).
+  This puts the capability gate at the credential layer, instead of a flag inside a
+  third-party server.
+- **Per-task tool profiles** mount only the servers that a given task's scope
+  requires.
+
+#### Rejected alternative: routed meta-tools / lazy tool registration
+
+Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large
+generated tool catalog. This is **rejected in favor of per-task tool profiles**:
+
+- *Two extra round-trips per call,* which is worse with a local model driving the
+  control loop.
+- *The model can no longer discover capabilities it wasn't already looking for.*
+  That is backwards for exploratory diagnostic work.
+- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON
+  Schema.
+- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what
+  makes a large static list affordable.
+
+Acceptable only when consuming a third-party server whose surface we can't trim.
+
+### F4. Containers, images, and restarts
 
 - **Containers + persistent volumes.** Stateless image; named volumes carry state
   across redeploys (and allow optional persistence between deployments).
@@ -907,6 +1019,9 @@ each agent is built — not standing up the service itself.
 - **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must
   never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the
   durability layers (A2/A3), not by restart timing.
+- **Carve-out: third-party MCP server images** (F3) are excluded from Watchtower.
+  They hold root-equivalent credentials to Proxmox and UniFi, so they are
+  digest-pinned and updated deliberately after review.
 - **Survive reboots:** VM + Docker service set to start on boot; `restart:
   unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval).
 - **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init,
@@ -976,8 +1091,11 @@ The rest resolve when reached, not now.
   are still moving. One repo can still build both agent images via path-filtered
   CI. Splitting later stays easy if a real need appears; merging later would not
   have been.
-- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The
+- **Sysadmin fallback browser access to network gear:** standing vs. per-task
-  lock-yourself-out risk argues for per-task.
+  hand-off. With UniFi reachable over MCP (F3), the browser is no longer the primary
+  path to network gear. It is a fallback for operations the UniFi API does not
+  expose. The open question is whether that *fallback* access is standing or handed
+  off per task. The lock-yourself-out risk still argues for per-task.
 - **Policy action taxonomy (C2):** the concrete list of which operations are
   allow vs. confirm. Best decided with the agent half-built in front of you.
 - **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
@@ -987,6 +1105,14 @@ The rest resolve when reached, not now.
   so boot-time reconciliation (A3) can re-derive gate state from Gitea even after
   state-volume loss — external truth wins (A2). Approval mechanics and revision
   binding are specified in A4.
+- **Parked: relocating stateful MCP servers off the owning agent's VM.** The trigger
+  is **statefulness** (a browser pool, a vector index, a warm cache), not sharing:
+  duplicating a stateful server duplicates its state. Relocating has a cost. A shared
+  server can't hold per-agent tokens, so per-agent identity has to move into the
+  request, through caller auth plus per-caller upstream credentials. That loses the
+  attribution property that makes instance-per-agent-VM (F3) clean.
+  Instance-per-agent-VM is the starting position. Migrate one server at a time, when
+  a specific server justifies it.
 
 ---
 
@@ -1013,6 +1139,7 @@ nisse-dev/               # app: sealed VM                        (Part B)
 nisse-ops/               # app: separate host, Tailscale         (Part C)
   policy engine, tiered confirmation, scheduled checks
   Vikunja ticket intake, Outline runbooks/records
+  MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM
   browser → admin UIs (API-first, UI fallback), confirm-first gating
 
 infra/                   # (future) Terraform + cloud-init       (Part F)