Record the MCP server deployment decisions in the spec #36
+146
-19
@@ -10,7 +10,10 @@ Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the
|
||||
document and no longer exists separately: its A-series components appear as A10–A16,
|
||||
its network and credential revisions in A5 and B3, and its revised build order as
|
||||
Part G. The inference-hardware question it raised has since been resolved in
|
||||
principle (Part H: a 4090-class host).
|
||||
principle (Part H: a 4090-class host). The MCP Deployment amendment has likewise been
|
||||
folded in: the every-agent-gets-a-VM rule, per-instance egress policy, and MCP server
|
||||
placement now form Part F1–F3; its consequences appear in Part 0 (site identifiers),
|
||||
A7, B3, C2, C3, C5, the Part F4 Watchtower carve-out, and Part H.
|
||||
|
||||
---
|
||||
|
||||
@@ -32,6 +35,12 @@ same string across all of those: the VM, the container, the Caddy host, the Matr
|
||||
account, and the Bitwarden machine account for a given agent all share one name, so
|
||||
that grepping the infrastructure for `nisse-ops` finds everything.
|
||||
|
||||
**Sites** are identified by their domain stem: `garrett-hall` (Arlington, hosts under
|
||||
|
dongar marked this conversation as resolved
Outdated
|
||||
`garrett-hall.com`) and `hartman-hut` (Cross Plains, hosts under `hartman-hut.com`).
|
||||
As with agent names, one string is used everywhere a site appears. For an MCP server
|
||||
instance (F3), that means its container name, its Vaultwarden machine account, and
|
||||
its MCP tool prefix.
|
||||
|
||||
The three artifacts live as three packages in **one repository, `nisse`** (decision
|
||||
recorded in Part H). Directory and package names carry the identity. If mirrored to
|
||||
GitHub, it goes under a personal account rather than an org — `github.com/nisse` is
|
||||
@@ -261,9 +270,13 @@ Requirements:
|
||||
Requirements:
|
||||
- Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or
|
||||
`bw` CLI + API key). Not the personal vault.
|
||||
- Reads only from a **dedicated, minimally-scoped machine account** per agent.
|
||||
- A single **bootstrap token** (the machine-account credential) is the one secret
|
||||
that lives on the box: tight file perms, never in image or repo.
|
||||
- Reads only from a **dedicated, minimally-scoped machine account** per agent, **plus
|
||||
one per MCP server instance** (F3). Each (service, site, privilege) instance holds
|
||||
its own upstream credential, so it gets its own machine account, and the agent's
|
||||
account cannot read an instance's credential.
|
||||
- Each machine account has a single **bootstrap token**. These are the only secrets
|
||||
that live on the box: tight file perms, never in image or repo. Each token is
|
||||
available only to its own container.
|
||||
- (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the
|
||||
interface able to grow toward this; don't build it yet.)
|
||||
|
||||
@@ -746,13 +759,16 @@ PR_OPEN
|
||||
|
||||
### B3. Deployment & Isolation
|
||||
|
||||
- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker
|
||||
containers, so it needs its own real Docker daemon. (Docker-in-a-container via
|
||||
socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the
|
||||
correct trust boundary for unattended arbitrary command execution.)
|
||||
- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no
|
||||
ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea
|
||||
IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM.
|
||||
- **Dedicated VM** (Proxmox), as for every agent (F1). The Dev Agent has an
|
||||
additional reason: it builds and runs Docker containers, so it needs its own real
|
||||
Docker daemon. (Docker-in-a-container via socket-mount ≈ host root; via dind =
|
||||
nested-storage pain. The VM boundary is the correct trust boundary for unattended
|
||||
arbitrary command execution.)
|
||||
- **Sealed to start:** `nisse-dev`'s egress allowlist (F2) permits LiteLLM, Gitea,
|
||||
and package registries, and there is **no ingress**. Egress denial must include
|
||||
**RFC1918 except the specific LiteLLM/Gitea IPs**, and also the tailnet's
|
||||
**`100.64.0.0/10`**, which is not RFC1918. Sealed from the internet must also mean
|
||||
no reach to NAS/Proxmox/UDM.
|
||||
- **The LAN-resident inference host is not a second exception.** It sits behind
|
||||
LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how
|
||||
many models or vendors are in play.
|
||||
@@ -789,9 +805,16 @@ deployments over the same library.
|
||||
|
||||
Tiered policy, evaluated on every action:
|
||||
- **Read-only telemetry** (metrics, logs, status) — free.
|
||||
- **Reversible service actions** (restart a container) — allowed.
|
||||
- **Reversible service actions** (restart a container) — allowed within an approved
|
||||
plan, without further confirmation.
|
||||
- **High-consequence actions** (storage, networking, auth) — **confirm-first** over
|
||||
transport before executing.
|
||||
transport before executing, even within an approved plan.
|
||||
|
||||
**Every write requires an approved plan first.** Write credentials exist only in the
|
||||
elevated MCP server instances (F3), and those are mounted only after plan approval. A
|
||||
reversible restart therefore needs an approved plan too. The tiers above govern
|
||||
execution *within* that plan. Read-only telemetry goes through the read-only
|
||||
instances, which are mounted by default, so it needs no plan.
|
||||
|
||||
This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
|
||||
irreversible / high-blast-radius actions.
|
||||
@@ -805,9 +828,13 @@ Special cases fold into the tiers:
|
||||
|
||||
### C3. Interfaces
|
||||
|
||||
- **MCP servers (F3):** Proxmox and UniFi, at both sites, through per-site,
|
||||
per-privilege server instances on the agent's own Docker host. This is the primary
|
||||
path to that infrastructure.
|
||||
- **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox,
|
||||
TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps —
|
||||
it's the least stable interface.
|
||||
TrueNAS good; UniFi now reachable over MCP); use the browser only for genuinely
|
||||
UI-only gaps. It's the least stable interface. For network gear, it is a fallback
|
||||
for API gaps, not the primary path.
|
||||
- **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value
|
||||
credential; its blast radius equals its Authelia scope, so scope it minimally.
|
||||
- **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden.
|
||||
@@ -835,8 +862,11 @@ The sysadmin analog of "dev agent driven by Gitea."
|
||||
|
||||
### C5. Deployment & Isolation
|
||||
|
||||
- Longer-lived, **stateful VM** (not disposable).
|
||||
- Longer-lived, **stateful VM** (not disposable), dedicated to this agent (F1).
|
||||
- Has **Tailscale** + scoped credentials to reach infrastructure across both sites.
|
||||
- **Its MCP servers (F3) run on its own Docker host.** Its egress allowlist (F2)
|
||||
therefore includes the Proxmox and UniFi endpoints at both sites, even though the
|
||||
agent process never opens those connections itself.
|
||||
- Updated **in place** via the container pipeline, **restart gated on an idle flag**.
|
||||
|
||||
---
|
||||
@@ -899,7 +929,89 @@ each agent is built — not standing up the service itself.
|
||||
|
||||
---
|
||||
|
||||
## PART F — DEPLOYMENT MODEL (both agents)
|
||||
## PART F — DEPLOYMENT MODEL (all agents)
|
||||
|
||||
These rules apply to every agent, current and future. Parts B3 and C5 record how each
|
||||
existing agent instantiates them.
|
||||
|
||||
### F1. Agent hosts: every agent gets its own VM
|
||||
|
||||
- **Every agent gets its own VM** (Proxmox). No agent runs as a container on a shared
|
||||
host, and no two agents share a VM.
|
||||
- **Why, in general:**
|
||||
- *Isolation boundary between agents.* The VM is the trust boundary (Part H). A
|
||||
compromised or misbehaving agent must not reach another agent's state,
|
||||
credentials, or processes.
|
||||
- *Per-agent credential attribution.* Everything an agent holds lives on its own
|
||||
host and is issued to its own identity, so every upstream action traces back to
|
||||
exactly one agent.
|
||||
- **Agent-specific reasons add to this; they are not its foundation.** The Dev Agent
|
||||
also needs a real Docker daemon, because it builds and runs containers (B3). An
|
||||
agent that builds no code still gets a VM, for the two reasons above.
|
||||
|
||||
### F2. Egress policy: per agent instance
|
||||
|
||||
- **Every agent instance has its own default-deny allowlist.** Requirements differ by
|
||||
agent type. No agent inherits another agent's policy.
|
||||
- **The allowlist is a declared artifact,** versioned alongside the agent's
|
||||
definition. Each entry is justified by the component that needs it.
|
||||
- **The declaration point is fixed. The enforcement point is not.** Policy may be
|
||||
enforced at the VM, at the Docker network, at the UniFi firewall, or at several of
|
||||
these at once. Layering is deliberate: enforcement at the network layer survives
|
||||
compromise of the agent's own host.
|
||||
- **The allowlist is the union** of what the agent reaches directly and what its MCP
|
||||
servers (F3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi
|
||||
endpoints at both sites, even though the agent never opens those connections
|
||||
itself.
|
||||
- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A
|
||||
rule written around RFC1918 denial does not cover it. Deny it explicitly.
|
||||
|
||||
### F3. MCP servers
|
||||
|
||||
**One instance per agent VM.** MCP servers run as containers on the owning agent's
|
||||
Docker host. Images are shared through the OCI registry. Processes are not shared
|
||||
between agents.
|
||||
|
||||
- *A shared process means a shared credential.* The upstream API's own audit log
|
||||
then cannot attribute a call to a specific agent. Separate instances with separate
|
||||
tokens give attribution at the API layer, instead of depending on our own
|
||||
transcripts. This matters most for a read-only auditing agent, such as a future
|
||||
`nisse-security`.
|
||||
- *An agent should not host the tooling used to inspect it.*
|
||||
- *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no
|
||||
state. (Stateful servers are the exception that could justify relocation. That
|
||||
item is parked in Part H.)
|
||||
|
||||
**One instance per (service, site, privilege level).**
|
||||
|
||||
- **Site belongs in the server's identity,** using the site identifier (Part 0), not
|
||||
in a `site` tool argument. A wrong-house action should require a tool that isn't
|
||||
mounted, not just a wrong parameter value.
|
||||
- **Read-only and elevated access are separate instances** with separate upstream
|
||||
credentials, each from its own machine account (A7). The read-only instance is
|
||||
mounted by default. The elevated instance is mounted only after plan approval (C2).
|
||||
This puts the capability gate at the credential layer, instead of a flag inside a
|
||||
third-party server.
|
||||
- **Per-task tool profiles** mount only the servers that a given task's scope
|
||||
requires.
|
||||
|
||||
#### Rejected alternative: routed meta-tools / lazy tool registration
|
||||
|
||||
Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large
|
||||
generated tool catalog. This is **rejected in favor of per-task tool profiles**:
|
||||
|
||||
- *Two extra round-trips per call,* which is worse with a local model driving the
|
||||
control loop.
|
||||
- *The model can no longer discover capabilities it wasn't already looking for.*
|
||||
That is backwards for exploratory diagnostic work.
|
||||
- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON
|
||||
Schema.
|
||||
- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what
|
||||
makes a large static list affordable.
|
||||
|
||||
Acceptable only when consuming a third-party server whose surface we can't trim.
|
||||
|
||||
### F4. Containers, images, and restarts
|
||||
|
||||
- **Containers + persistent volumes.** Stateless image; named volumes carry state
|
||||
across redeploys (and allow optional persistence between deployments).
|
||||
@@ -907,6 +1019,9 @@ each agent is built — not standing up the service itself.
|
||||
- **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must
|
||||
never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the
|
||||
durability layers (A2/A3), not by restart timing.
|
||||
- **Carve-out: third-party MCP server images** (F3) are excluded from Watchtower.
|
||||
They hold root-equivalent credentials to Proxmox and UniFi, so they are
|
||||
digest-pinned and updated deliberately after review.
|
||||
- **Survive reboots:** VM + Docker service set to start on boot; `restart:
|
||||
unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval).
|
||||
- **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init,
|
||||
@@ -976,8 +1091,11 @@ The rest resolve when reached, not now.
|
||||
are still moving. One repo can still build both agent images via path-filtered
|
||||
CI. Splitting later stays easy if a real need appears; merging later would not
|
||||
have been.
|
||||
- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The
|
||||
lock-yourself-out risk argues for per-task.
|
||||
- **Sysadmin fallback browser access to network gear:** standing vs. per-task
|
||||
hand-off. With UniFi reachable over MCP (F3), the browser is no longer the primary
|
||||
path to network gear. It is a fallback for operations the UniFi API does not
|
||||
expose. The open question is whether that *fallback* access is standing or handed
|
||||
off per task. The lock-yourself-out risk still argues for per-task.
|
||||
- **Policy action taxonomy (C2):** the concrete list of which operations are
|
||||
allow vs. confirm. Best decided with the agent half-built in front of you.
|
||||
- **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
|
||||
@@ -987,6 +1105,14 @@ The rest resolve when reached, not now.
|
||||
so boot-time reconciliation (A3) can re-derive gate state from Gitea even after
|
||||
state-volume loss — external truth wins (A2). Approval mechanics and revision
|
||||
binding are specified in A4.
|
||||
- **Parked: relocating stateful MCP servers off the owning agent's VM.** The trigger
|
||||
is **statefulness** (a browser pool, a vector index, a warm cache), not sharing:
|
||||
duplicating a stateful server duplicates its state. Relocating has a cost. A shared
|
||||
server can't hold per-agent tokens, so per-agent identity has to move into the
|
||||
request, through caller auth plus per-caller upstream credentials. That loses the
|
||||
attribution property that makes instance-per-agent-VM (F3) clean.
|
||||
Instance-per-agent-VM is the starting position. Migrate one server at a time, when
|
||||
a specific server justifies it.
|
||||
|
||||
---
|
||||
|
||||
@@ -1013,6 +1139,7 @@ nisse-dev/ # app: sealed VM (Part B)
|
||||
nisse-ops/ # app: separate host, Tailscale (Part C)
|
||||
policy engine, tiered confirmation, scheduled checks
|
||||
Vikunja ticket intake, Outline runbooks/records
|
||||
MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM
|
||||
browser → admin UIs (API-first, UI fallback), confirm-first gating
|
||||
|
||||
infra/ # (future) Terraform + cloud-init (Part F)
|
||||
|
||||
Reference in new issue
Block a user
One note (and I think this was wrong in the original prompt), the domains for the two sites end in garrett-hall.com, and hartman-hut.com.
Good catch. The identifiers themselves stay the same:
garrett-hallandhartman-hutare the stems ofgarrett-hall.comandhartman-hut.com. But the text said "domain stem" without naming the domains, so a reader could not check which domain it meant. Each site now names its domain: "hosts undergarrett-hall.com" and "hosts underhartman-hut.com".