Record the MCP server deployment decisions in the spec #36

Merged
dongar merged 3 commits from mcp-deployment into main 2026-09-23 02:03:59 +00:00
+146 -19
View File
@@ -10,7 +10,10 @@ Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the
document and no longer exists separately: its A-series components appear as A10–A16,
its network and credential revisions in A5 and B3, and its revised build order as
Part G. The inference-hardware question it raised has since been resolved in
principle (Part H: a 4090-class host).
principle (Part H: a 4090-class host). The MCP Deployment amendment has likewise been
folded in: the every-agent-gets-a-VM rule, per-instance egress policy, and MCP server
placement now form Part F1–F3; its consequences appear in Part 0 (site identifiers),
A7, B3, C2, C3, C5, the Part F4 Watchtower carve-out, and Part H.
---
@@ -32,6 +35,12 @@ same string across all of those: the VM, the container, the Caddy host, the Matr
account, and the Bitwarden machine account for a given agent all share one name, so
that grepping the infrastructure for `nisse-ops` finds everything.
**Sites** are identified by their domain stem: `garrett-hall` (Arlington, hosts under
dongar marked this conversation as resolved Outdated
Outdated
Review

One note (and I think this was wrong in the original prompt), the domains for the two sites end in garrett-hall.com, and hartman-hut.com.

One note (and I think this was wrong in the original prompt), the domains for the two sites end in garrett-hall.com, and hartman-hut.com.
Outdated
Review

Good catch. The identifiers themselves stay the same: garrett-hall and hartman-hut are the stems of garrett-hall.com and hartman-hut.com. But the text said "domain stem" without naming the domains, so a reader could not check which domain it meant. Each site now names its domain: "hosts under garrett-hall.com" and "hosts under hartman-hut.com".

Good catch. The identifiers themselves stay the same: `garrett-hall` and `hartman-hut` are the stems of `garrett-hall.com` and `hartman-hut.com`. But the text said "domain stem" without naming the domains, so a reader could not check which domain it meant. Each site now names its domain: "hosts under `garrett-hall.com`" and "hosts under `hartman-hut.com`".
`garrett-hall.com`) and `hartman-hut` (Cross Plains, hosts under `hartman-hut.com`).
As with agent names, one string is used everywhere a site appears. For an MCP server
instance (F3), that means its container name, its Vaultwarden machine account, and
its MCP tool prefix.
The three artifacts live as three packages in **one repository, `nisse`** (decision
recorded in Part H). Directory and package names carry the identity. If mirrored to
GitHub, it goes under a personal account rather than an org — `github.com/nisse` is
@@ -261,9 +270,13 @@ Requirements:
Requirements:
- Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or
`bw` CLI + API key). Not the personal vault.
- Reads only from a **dedicated, minimally-scoped machine account** per agent.
- A single **bootstrap token** (the machine-account credential) is the one secret
that lives on the box: tight file perms, never in image or repo.
- Reads only from a **dedicated, minimally-scoped machine account** per agent, **plus
one per MCP server instance** (F3). Each (service, site, privilege) instance holds
its own upstream credential, so it gets its own machine account, and the agent's
account cannot read an instance's credential.
- Each machine account has a single **bootstrap token**. These are the only secrets
that live on the box: tight file perms, never in image or repo. Each token is
available only to its own container.
- (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the
interface able to grow toward this; don't build it yet.)
@@ -746,13 +759,16 @@ PR_OPEN
### B3. Deployment & Isolation
- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker
containers, so it needs its own real Docker daemon. (Docker-in-a-container via
socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the
correct trust boundary for unattended arbitrary command execution.)
- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no
ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea
IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM.
- **Dedicated VM** (Proxmox), as for every agent (F1). The Dev Agent has an
additional reason: it builds and runs Docker containers, so it needs its own real
Docker daemon. (Docker-in-a-container via socket-mount ≈ host root; via dind =
nested-storage pain. The VM boundary is the correct trust boundary for unattended
arbitrary command execution.)
- **Sealed to start:** `nisse-dev`'s egress allowlist (F2) permits LiteLLM, Gitea,
and package registries, and there is **no ingress**. Egress denial must include
**RFC1918 except the specific LiteLLM/Gitea IPs**, and also the tailnet's
**`100.64.0.0/10`**, which is not RFC1918. Sealed from the internet must also mean
no reach to NAS/Proxmox/UDM.
- **The LAN-resident inference host is not a second exception.** It sits behind
LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how
many models or vendors are in play.
@@ -789,9 +805,16 @@ deployments over the same library.
Tiered policy, evaluated on every action:
- **Read-only telemetry** (metrics, logs, status) — free.
- **Reversible service actions** (restart a container) — allowed.
- **Reversible service actions** (restart a container) — allowed within an approved
plan, without further confirmation.
- **High-consequence actions** (storage, networking, auth) — **confirm-first** over
transport before executing.
transport before executing, even within an approved plan.
**Every write requires an approved plan first.** Write credentials exist only in the
elevated MCP server instances (F3), and those are mounted only after plan approval. A
reversible restart therefore needs an approved plan too. The tiers above govern
execution *within* that plan. Read-only telemetry goes through the read-only
instances, which are mounted by default, so it needs no plan.
This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
irreversible / high-blast-radius actions.
@@ -805,9 +828,13 @@ Special cases fold into the tiers:
### C3. Interfaces
- **MCP servers (F3):** Proxmox and UniFi, at both sites, through per-site,
per-privilege server instances on the agent's own Docker host. This is the primary
path to that infrastructure.
- **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox,
TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps —
it's the least stable interface.
TrueNAS good; UniFi now reachable over MCP); use the browser only for genuinely
UI-only gaps. It's the least stable interface. For network gear, it is a fallback
for API gaps, not the primary path.
- **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value
credential; its blast radius equals its Authelia scope, so scope it minimally.
- **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden.
@@ -835,8 +862,11 @@ The sysadmin analog of "dev agent driven by Gitea."
### C5. Deployment & Isolation
- Longer-lived, **stateful VM** (not disposable).
- Longer-lived, **stateful VM** (not disposable), dedicated to this agent (F1).
- Has **Tailscale** + scoped credentials to reach infrastructure across both sites.
- **Its MCP servers (F3) run on its own Docker host.** Its egress allowlist (F2)
therefore includes the Proxmox and UniFi endpoints at both sites, even though the
agent process never opens those connections itself.
- Updated **in place** via the container pipeline, **restart gated on an idle flag**.
---
@@ -899,7 +929,89 @@ each agent is built — not standing up the service itself.
---
## PART F — DEPLOYMENT MODEL (both agents)
## PART F — DEPLOYMENT MODEL (all agents)
These rules apply to every agent, current and future. Parts B3 and C5 record how each
existing agent instantiates them.
### F1. Agent hosts: every agent gets its own VM
- **Every agent gets its own VM** (Proxmox). No agent runs as a container on a shared
host, and no two agents share a VM.
- **Why, in general:**
- *Isolation boundary between agents.* The VM is the trust boundary (Part H). A
compromised or misbehaving agent must not reach another agent's state,
credentials, or processes.
- *Per-agent credential attribution.* Everything an agent holds lives on its own
host and is issued to its own identity, so every upstream action traces back to
exactly one agent.
- **Agent-specific reasons add to this; they are not its foundation.** The Dev Agent
also needs a real Docker daemon, because it builds and runs containers (B3). An
agent that builds no code still gets a VM, for the two reasons above.
### F2. Egress policy: per agent instance
- **Every agent instance has its own default-deny allowlist.** Requirements differ by
agent type. No agent inherits another agent's policy.
- **The allowlist is a declared artifact,** versioned alongside the agent's
definition. Each entry is justified by the component that needs it.
- **The declaration point is fixed. The enforcement point is not.** Policy may be
enforced at the VM, at the Docker network, at the UniFi firewall, or at several of
these at once. Layering is deliberate: enforcement at the network layer survives
compromise of the agent's own host.
- **The allowlist is the union** of what the agent reaches directly and what its MCP
servers (F3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi
endpoints at both sites, even though the agent never opens those connections
itself.
- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A
rule written around RFC1918 denial does not cover it. Deny it explicitly.
### F3. MCP servers
**One instance per agent VM.** MCP servers run as containers on the owning agent's
Docker host. Images are shared through the OCI registry. Processes are not shared
between agents.
- *A shared process means a shared credential.* The upstream API's own audit log
then cannot attribute a call to a specific agent. Separate instances with separate
tokens give attribution at the API layer, instead of depending on our own
transcripts. This matters most for a read-only auditing agent, such as a future
`nisse-security`.
- *An agent should not host the tooling used to inspect it.*
- *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no
state. (Stateful servers are the exception that could justify relocation. That
item is parked in Part H.)
**One instance per (service, site, privilege level).**
- **Site belongs in the server's identity,** using the site identifier (Part 0), not
in a `site` tool argument. A wrong-house action should require a tool that isn't
mounted, not just a wrong parameter value.
- **Read-only and elevated access are separate instances** with separate upstream
credentials, each from its own machine account (A7). The read-only instance is
mounted by default. The elevated instance is mounted only after plan approval (C2).
This puts the capability gate at the credential layer, instead of a flag inside a
third-party server.
- **Per-task tool profiles** mount only the servers that a given task's scope
requires.
#### Rejected alternative: routed meta-tools / lazy tool registration
Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large
generated tool catalog. This is **rejected in favor of per-task tool profiles**:
- *Two extra round-trips per call,* which is worse with a local model driving the
control loop.
- *The model can no longer discover capabilities it wasn't already looking for.*
That is backwards for exploratory diagnostic work.
- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON
Schema.
- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what
makes a large static list affordable.
Acceptable only when consuming a third-party server whose surface we can't trim.
### F4. Containers, images, and restarts
- **Containers + persistent volumes.** Stateless image; named volumes carry state
across redeploys (and allow optional persistence between deployments).
@@ -907,6 +1019,9 @@ each agent is built — not standing up the service itself.
- **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must
never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the
durability layers (A2/A3), not by restart timing.
- **Carve-out: third-party MCP server images** (F3) are excluded from Watchtower.
They hold root-equivalent credentials to Proxmox and UniFi, so they are
digest-pinned and updated deliberately after review.
- **Survive reboots:** VM + Docker service set to start on boot; `restart:
unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval).
- **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init,
@@ -976,8 +1091,11 @@ The rest resolve when reached, not now.
are still moving. One repo can still build both agent images via path-filtered
CI. Splitting later stays easy if a real need appears; merging later would not
have been.
- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The
lock-yourself-out risk argues for per-task.
- **Sysadmin fallback browser access to network gear:** standing vs. per-task
hand-off. With UniFi reachable over MCP (F3), the browser is no longer the primary
path to network gear. It is a fallback for operations the UniFi API does not
expose. The open question is whether that *fallback* access is standing or handed
off per task. The lock-yourself-out risk still argues for per-task.
- **Policy action taxonomy (C2):** the concrete list of which operations are
allow vs. confirm. Best decided with the agent half-built in front of you.
- **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
@@ -987,6 +1105,14 @@ The rest resolve when reached, not now.
so boot-time reconciliation (A3) can re-derive gate state from Gitea even after
state-volume loss — external truth wins (A2). Approval mechanics and revision
binding are specified in A4.
- **Parked: relocating stateful MCP servers off the owning agent's VM.** The trigger
is **statefulness** (a browser pool, a vector index, a warm cache), not sharing:
duplicating a stateful server duplicates its state. Relocating has a cost. A shared
server can't hold per-agent tokens, so per-agent identity has to move into the
request, through caller auth plus per-caller upstream credentials. That loses the
attribution property that makes instance-per-agent-VM (F3) clean.
Instance-per-agent-VM is the starting position. Migrate one server at a time, when
a specific server justifies it.
---
@@ -1013,6 +1139,7 @@ nisse-dev/ # app: sealed VM (Part B)
nisse-ops/ # app: separate host, Tailscale (Part C)
policy engine, tiered confirmation, scheduled checks
Vikunja ticket intake, Outline runbooks/records
MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM
browser → admin UIs (API-first, UI fallback), confirm-first gating
infra/ # (future) Terraform + cloud-init (Part F)