diff --git a/docs/Architecture-Build-Spec.md b/docs/Architecture-Build-Spec.md index 9c43c9a..74bb959 100644 --- a/docs/Architecture-Build-Spec.md +++ b/docs/Architecture-Build-Spec.md @@ -10,7 +10,10 @@ Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the document and no longer exists separately: its A-series components appear as A10–A16, its network and credential revisions in A5 and B3, and its revised build order as Part G. The inference-hardware question it raised has since been resolved in -principle (Part H: a 4090-class host). +principle (Part H: a 4090-class host). The MCP Deployment amendment has likewise been +folded in: the every-agent-gets-a-VM rule, per-instance egress policy, and MCP server +placement now form Part F1–F3; its consequences appear in Part 0 (site identifiers), +A7, B3, C2, C3, C5, the Part F4 Watchtower carve-out, and Part H. --- @@ -32,6 +35,12 @@ same string across all of those: the VM, the container, the Caddy host, the Matr account, and the Bitwarden machine account for a given agent all share one name, so that grepping the infrastructure for `nisse-ops` finds everything. +**Sites** are identified by their domain stem: `garrett-hall` (Arlington, hosts under +`garrett-hall.com`) and `hartman-hut` (Cross Plains, hosts under `hartman-hut.com`). +As with agent names, one string is used everywhere a site appears. For an MCP server +instance (F3), that means its container name, its Vaultwarden machine account, and +its MCP tool prefix. + The three artifacts live as three packages in **one repository, `nisse`** (decision recorded in Part H). Directory and package names carry the identity. If mirrored to GitHub, it goes under a personal account rather than an org — `github.com/nisse` is @@ -261,9 +270,13 @@ Requirements: Requirements: - Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or `bw` CLI + API key). Not the personal vault. -- Reads only from a **dedicated, minimally-scoped machine account** per agent. -- A single **bootstrap token** (the machine-account credential) is the one secret - that lives on the box: tight file perms, never in image or repo. +- Reads only from a **dedicated, minimally-scoped machine account** per agent, **plus + one per MCP server instance** (F3). Each (service, site, privilege) instance holds + its own upstream credential, so it gets its own machine account, and the agent's + account cannot read an instance's credential. +- Each machine account has a single **bootstrap token**. These are the only secrets + that live on the box: tight file perms, never in image or repo. Each token is + available only to its own container. - (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the interface able to grow toward this; don't build it yet.) @@ -746,13 +759,16 @@ PR_OPEN ### B3. Deployment & Isolation -- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker - containers, so it needs its own real Docker daemon. (Docker-in-a-container via - socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the - correct trust boundary for unattended arbitrary command execution.) -- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no - ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea - IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM. +- **Dedicated VM** (Proxmox), as for every agent (F1). The Dev Agent has an + additional reason: it builds and runs Docker containers, so it needs its own real + Docker daemon. (Docker-in-a-container via socket-mount ≈ host root; via dind = + nested-storage pain. The VM boundary is the correct trust boundary for unattended + arbitrary command execution.) +- **Sealed to start:** `nisse-dev`'s egress allowlist (F2) permits LiteLLM, Gitea, + and package registries, and there is **no ingress**. Egress denial must include + **RFC1918 except the specific LiteLLM/Gitea IPs**, and also the tailnet's + **`100.64.0.0/10`**, which is not RFC1918. Sealed from the internet must also mean + no reach to NAS/Proxmox/UDM. - **The LAN-resident inference host is not a second exception.** It sits behind LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how many models or vendors are in play. @@ -789,9 +805,16 @@ deployments over the same library. Tiered policy, evaluated on every action: - **Read-only telemetry** (metrics, logs, status) — free. -- **Reversible service actions** (restart a container) — allowed. +- **Reversible service actions** (restart a container) — allowed within an approved + plan, without further confirmation. - **High-consequence actions** (storage, networking, auth) — **confirm-first** over - transport before executing. + transport before executing, even within an approved plan. + +**Every write requires an approved plan first.** Write credentials exist only in the +elevated MCP server instances (F3), and those are mounted only after plan approval. A +reversible restart therefore needs an approved plan too. The tiers above govern +execution *within* that plan. Read-only telemetry goes through the read-only +instances, which are mounted by default, so it needs no plan. This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at irreversible / high-blast-radius actions. @@ -805,9 +828,13 @@ Special cases fold into the tiers: ### C3. Interfaces +- **MCP servers (F3):** Proxmox and UniFi, at both sites, through per-site, + per-privilege server instances on the agent's own Docker host. This is the primary + path to that infrastructure. - **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox, - TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps — - it's the least stable interface. + TrueNAS good; UniFi now reachable over MCP); use the browser only for genuinely + UI-only gaps. It's the least stable interface. For network gear, it is a fallback + for API gaps, not the primary path. - **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value credential; its blast radius equals its Authelia scope, so scope it minimally. - **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden. @@ -835,8 +862,11 @@ The sysadmin analog of "dev agent driven by Gitea." ### C5. Deployment & Isolation -- Longer-lived, **stateful VM** (not disposable). +- Longer-lived, **stateful VM** (not disposable), dedicated to this agent (F1). - Has **Tailscale** + scoped credentials to reach infrastructure across both sites. +- **Its MCP servers (F3) run on its own Docker host.** Its egress allowlist (F2) + therefore includes the Proxmox and UniFi endpoints at both sites, even though the + agent process never opens those connections itself. - Updated **in place** via the container pipeline, **restart gated on an idle flag**. --- @@ -899,7 +929,89 @@ each agent is built — not standing up the service itself. --- -## PART F — DEPLOYMENT MODEL (both agents) +## PART F — DEPLOYMENT MODEL (all agents) + +These rules apply to every agent, current and future. Parts B3 and C5 record how each +existing agent instantiates them. + +### F1. Agent hosts: every agent gets its own VM + +- **Every agent gets its own VM** (Proxmox). No agent runs as a container on a shared + host, and no two agents share a VM. +- **Why, in general:** + - *Isolation boundary between agents.* The VM is the trust boundary (Part H). A + compromised or misbehaving agent must not reach another agent's state, + credentials, or processes. + - *Per-agent credential attribution.* Everything an agent holds lives on its own + host and is issued to its own identity, so every upstream action traces back to + exactly one agent. +- **Agent-specific reasons add to this; they are not its foundation.** The Dev Agent + also needs a real Docker daemon, because it builds and runs containers (B3). An + agent that builds no code still gets a VM, for the two reasons above. + +### F2. Egress policy: per agent instance + +- **Every agent instance has its own default-deny allowlist.** Requirements differ by + agent type. No agent inherits another agent's policy. +- **The allowlist is a declared artifact,** versioned alongside the agent's + definition. Each entry is justified by the component that needs it. +- **The declaration point is fixed. The enforcement point is not.** Policy may be + enforced at the VM, at the Docker network, at the UniFi firewall, or at several of + these at once. Layering is deliberate: enforcement at the network layer survives + compromise of the agent's own host. +- **The allowlist is the union** of what the agent reaches directly and what its MCP + servers (F3) reach on its behalf. `nisse-ops` must permit Proxmox and UniFi + endpoints at both sites, even though the agent never opens those connections + itself. +- **The tailnet is `100.64.0.0/10`** (CGNAT shared address space), not RFC1918. A + rule written around RFC1918 denial does not cover it. Deny it explicitly. + +### F3. MCP servers + +**One instance per agent VM.** MCP servers run as containers on the owning agent's +Docker host. Images are shared through the OCI registry. Processes are not shared +between agents. + +- *A shared process means a shared credential.* The upstream API's own audit log + then cannot attribute a call to a specific agent. Separate instances with separate + tokens give attribution at the API layer, instead of depending on our own + transcripts. This matters most for a read-only auditing agent, such as a future + `nisse-security`. +- *An agent should not host the tooling used to inspect it.* +- *Duplication is nearly free.* These servers wrap remote HTTP APIs and hold no + state. (Stateful servers are the exception that could justify relocation. That + item is parked in Part H.) + +**One instance per (service, site, privilege level).** + +- **Site belongs in the server's identity,** using the site identifier (Part 0), not + in a `site` tool argument. A wrong-house action should require a tool that isn't + mounted, not just a wrong parameter value. +- **Read-only and elevated access are separate instances** with separate upstream + credentials, each from its own machine account (A7). The read-only instance is + mounted by default. The elevated instance is mounted only after plan approval (C2). + This puts the capability gate at the credential layer, instead of a flag inside a + third-party server. +- **Per-task tool profiles** mount only the servers that a given task's scope + requires. + +#### Rejected alternative: routed meta-tools / lazy tool registration + +Several community Proxmox and UniFi MCP servers use search-then-dispatch over a large +generated tool catalog. This is **rejected in favor of per-task tool profiles**: + +- *Two extra round-trips per call,* which is worse with a local model driving the + control loop. +- *The model can no longer discover capabilities it wasn't already looking for.* + That is backwards for exploratory diagnostic work. +- *Arguments arrive as an opaque blob,* instead of being constrained by per-tool JSON + Schema. +- *A dynamic tool list defeats prompt prefix caching,* and caching is much of what + makes a large static list affordable. + +Acceptable only when consuming a third-party server whose surface we can't trim. + +### F4. Containers, images, and restarts - **Containers + persistent volumes.** Stateless image; named volumes carry state across redeploys (and allow optional persistence between deployments). @@ -907,6 +1019,9 @@ each agent is built — not standing up the service itself. - **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the durability layers (A2/A3), not by restart timing. +- **Carve-out: third-party MCP server images** (F3) are excluded from Watchtower. + They hold root-equivalent credentials to Proxmox and UniFi, so they are + digest-pinned and updated deliberately after review. - **Survive reboots:** VM + Docker service set to start on boot; `restart: unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval). - **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init, @@ -976,8 +1091,11 @@ The rest resolve when reached, not now. are still moving. One repo can still build both agent images via path-filtered CI. Splitting later stays easy if a real need appears; merging later would not have been. -- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The - lock-yourself-out risk argues for per-task. +- **Sysadmin fallback browser access to network gear:** standing vs. per-task + hand-off. With UniFi reachable over MCP (F3), the browser is no longer the primary + path to network gear. It is a fallback for operations the UniFi API does not + expose. The open question is whether that *fallback* access is standing or handed + off per task. The lock-yourself-out risk still argues for per-task. - **Policy action taxonomy (C2):** the concrete list of which operations are allow vs. confirm. Best decided with the agent half-built in front of you. - **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are @@ -987,6 +1105,14 @@ The rest resolve when reached, not now. so boot-time reconciliation (A3) can re-derive gate state from Gitea even after state-volume loss — external truth wins (A2). Approval mechanics and revision binding are specified in A4. +- **Parked: relocating stateful MCP servers off the owning agent's VM.** The trigger + is **statefulness** (a browser pool, a vector index, a warm cache), not sharing: + duplicating a stateful server duplicates its state. Relocating has a cost. A shared + server can't hold per-agent tokens, so per-agent identity has to move into the + request, through caller auth plus per-caller upstream credentials. That loses the + attribution property that makes instance-per-agent-VM (F3) clean. + Instance-per-agent-VM is the starting position. Migrate one server at a time, when + a specific server justifies it. --- @@ -1013,6 +1139,7 @@ nisse-dev/ # app: sealed VM (Part B) nisse-ops/ # app: separate host, Tailscale (Part C) policy engine, tiered confirmation, scheduled checks Vikunja ticket intake, Outline runbooks/records + MCP → Proxmox/UniFi per site and privilege (F3); runs on its own VM browser → admin UIs (API-first, UI fallback), confirm-first gating infra/ # (future) Terraform + cloud-init (Part F)