nisse/docs/Architecture-Build-Spec.md

# Nisse — Build Specification

A build-oriented spec for two self-hosted AI agents sharing a common library. This
document states **requirements per component** so major pieces can be broken out and
built independently. It is deliberately lighter on the supporting services (Matrix,
Gitea, Vikunja, Outline, LiteLLM) and heaviest on the **agent library** and the
**two agents**, which is where the real design work lives.

**Document history.** The Tiered Inference amendment has been folded into this
document and no longer exists separately: its A-series components appear as A10–A16,
its network and credential revisions in A5 and B3, and its revised build order as
Part G. The inference-hardware question it raised has since been resolved in
principle (Part H: a 4090-class host).

---

## 0. Naming

The project is **Nisse**, after the Nordic farm spirit that keeps the place running
overnight and expects to be treated well in return. Three artifacts:

| Artifact | Name | Role |
|---|---|---|
| Shared library | `nisse-core` | The spine both agents sit on (Part A) |
| Dev Agent | `nisse-dev` | Plans and implements software projects (Part B) |
| Sysadmin Agent | `nisse-ops` | Operates infrastructure (Part C) |

Throughout this document, **Dev Agent** and **Sysadmin Agent** name the *roles* in
conceptual discussion; `nisse-dev` and `nisse-ops` name the *artifacts* — repos,
container images, VM hostnames, Matrix identities, and service accounts. Use the
same string across all of those: the VM, the container, the Caddy host, the Matrix
account, and the Bitwarden machine account for a given agent all share one name, so
that grepping the infrastructure for `nisse-ops` finds everything.

Published under a personal GitHub account rather than an org; `github.com/nisse` is
squatted and GitHub does not reclaim inactive usernames. The repo names carry the
identity. `nisse-core` is free on PyPI if packages are ever published.

---

## 0.1 What We're Building

Two agents:

1. **Dev Agent** — `nisse-dev` (build first) — plans and implements software
   projects, produces Gitea PRs for human review, reacts to review comments and
   commits.
2. **Sysadmin Agent** — `nisse-ops` (follow-on) — operates and maintains
   infrastructure, driven by tickets, gated on high-consequence actions.

Both are thin applications over a shared **`nisse-core`** library. The library is
*extracted from the Dev Agent after it works* — not designed up front for two
consumers that don't exist yet. Build the Dev Agent as if the library were internal,
then lift the seams that proved real.

Whether the three artifacts live in one repository or three is **an open decision**
(Part H). It changes what the extraction step in the build order concretely means.

### Guiding principles (apply to everything below)

- **Human-gated autonomy.** Agents work independently once told to start, but never
  cross key boundaries (start implementation, apply infrastructure change) without
  explicit sign-off.
- **No per-command approval.** Within an approved unit of work, execution is
  unattended. Control lives at the *boundaries*, not per action.
- **Isolation matched to blast radius.** Dev Agent is disposable and sealed; Sysadmin
  Agent is trusted with live-infrastructure reach. They never share a running
  instance.
- **Stateless image, stateful volumes.** Containers are replaceable; durable state
  lives on named volumes and must survive both redeploys and unexpected reboots.
- **Structured channels for machines, prose channels for humans.** Never route
  machine event intake through human-facing formatters (chat bridges).
- **Own the stack.** Everything self-hostable stays inside the network where practical.

---

## PART A — THE AGENT LIBRARY (`nisse-core`)

The shared spine. Every component here is consumed by both agents. This is the part
to get right.

### A1. Transport

**Purpose:** all human⇄agent conversation — commands in, questions/approvals/updates
out.

Requirements:
- A `Transport` interface the orchestrator depends on; **no platform SDK imported
  directly** anywhere else. Methods (at least): `send_message`,
  `send_approval_request`, `ask_question`, `on_message` (inbound callback/stream).
- **`MatrixTransport`** is the first and primary implementation. Self-hosted
  homeserver (Synapse or Conduit) keeps the control plane in-network. Must support
  threaded conversations (for plan discussion) and per-agent identities.
- Interface must make adding `TelegramTransport` / `SlackTransport` later a matter of
  one new class + config, no orchestrator changes.
- Both agents run as **distinct identities** (`@nisse-dev`, `@nisse-ops`) in separate
  rooms on the same homeserver.

Explicitly *not* required: rich message formatting, custom clients. Existing Matrix
clients (Element) are the UI.

**Matrix is strictly a UI channel.** Although the homeserver is self-hosted and
therefore not "remote," room history must not become load-bearing context. The
canonical log (A10) is the sole source of truth.

### A2. State Store and Durability *(critical path — build first)*

**Purpose:** durable, crash-safe persistence of everything that must outlive a
process restart. This is what makes the whole system safe to redeploy and reboot; it
blocks everything else.

Must persist: active project/task, per-project state-machine position, approvals
granted, pending questions, session continuity, (dev) git clone locations.

Disposable (never on the state volume): agent code, build artifacts/test containers,
shipped-out logs.

Requirements:
- **Crash-atomic writes.** SQLite in **WAL mode** (default choice) or atomic
  file writes (`write-temp → fsync → rename`). A power cut mid-write must never
  produce torn state.
- **Checkpoint after every state transition**, with `fsync`. No "flush later" —
  in-memory state that hasn't hit disk is assumed lost.
- **Durable intent log.** Before a unit of work starts, record intent, so a restart
  knows what was in flight.
- **Idempotent transitions reconciled against external truth.** On any operation with
  an external side effect (open PR, push branch), restart must detect the effect
  already happened (query Gitea/git) rather than repeat it.
- **`schema_version`** field from the first commit, with a migration hook, so a newer
  image can read older state.

### A3. Boot-Time Reconciliation

**Purpose:** turn an ambiguous crash into a safe, known state.

Every startup (clean or post-crash), before accepting operator commands:
1. Open state store (WAL recovery is automatic).
2. Run any pending schema migration.
3. Read intent log; look for a transition that began but never committed.
4. Reconcile against external truth (Gitea/git for the dev agent; relevant services
   for the sysadmin agent).
5. **Interrupted mid-delegation (A13) — apply the deterministic resume rule.**
   **Auto-resume** the delegation — re-render its transcript from the canonical log
   and continue the loop, posting a notice (not a question) to transport — when the
   cut is clean: the last durable event is a completed `tool_result` or assistant
   turn, the delegation branch sits at the expected commit, no cap is breached, and
   external truth matches the log. **Gate to `AWAIT_INPUT`** when the last event is a
   `tool_call` with no recorded result whose effect cannot be verified by inspection
   (tree diff for edits, Gitea query for pushes), or on any branch/log mismatch. The
   judgment is an observable check, never model self-assessment — the same reasoning
   as A12.
6. Any other interruption → move to **`AWAIT_INPUT`** and post to transport
   (e.g. "restarted mid-milestone-2, branch at commit abc123, continue or re-plan?").
   Never blind-resume.
7. Otherwise resume the last committed state.

### A4. Control Loop, State Machine, and Gating Primitives

**Purpose:** the generic engine that drives an agent through gated work. Each agent
supplies its own concrete states; the primitives are shared.

Requirements:
- A **state machine** framework: named states, transitions, and human gates. Each
  gate blocks until an explicit operator action arrives via transport.
- **Gating is structural, not advisory.** The mechanism (used by both agents):
  - Expose a **`submit_plan(type, content)`** tool to the model. Calling it posts the
    plan for approval and **ends the agent's turn.** The agent cannot self-advance.
  - Expose an **`ask_human(question)`** tool — non-terminal: pause, post question,
    block until answered, resume.
  - **Do not** expose any "proceed" / "start" tool. Advancement past a gate happens
    only when the orchestrator receives operator approval and hands over the next
    task. The agent has no path to approve itself.
- **Approval sources.** A gate waits on an approval event from any registered
  **`ApprovalSource`** — the Matrix transport (A1), the Gitea comment poller (B2), or
  the Vikunja comment stream (C4). First valid approval wins and is consumed; the
  gate records which channel satisfied it, and the orchestrator echoes the outcome to
  the other channels so the records converge. Each source authenticates the approver
  against a per-channel operator allowlist (Matrix MXID, Gitea username, Vikunja
  username).
- **Approvals bind to a plan revision, never to "the current gate."** Every submitted
  plan revision carries a monotonically-increasing tag (`rev-N`). An approval naming
  a stale revision is rejected with a notice, never applied to a newer one — this is
  what makes multi-channel approval race-free.
- **One consistent approval UI across all channels.** Approval is always the typed
  word `approve`, attached to a specific revision: in Matrix, a reply to the plan
  message (the reply relation carries the binding; a bare `approve` in the room binds
  to the latest revision only when unambiguous, otherwise the agent asks); in Gitea
  and Vikunja, a comment `approve rev-N`, with the tag printed in the plan comment
  for copy-paste. Any other reply/comment is `reject <feedback>`. Reactions/emoji are
  never an approval path. Approvals are consumed at first sight; later edits to an
  approval comment are ignored and logged.
- **Autonomy inside an approved unit of work:** all tool/command use auto-approved
  (Claude Agent SDK `bypassPermissions` or an allow-all `canUseTool`). Escalation to
  the human happens *only* via `ask_human`.
- The loop must survive restart mid-work (relies on A2/A3).

### A5. LLM Access *(tiered)*

**Purpose:** reach both a locally-hosted model and frontier models through one
interface, with the frontier vendor swappable by config.

A locally-hosted model drives the control loop and performs mechanical work; frontier
models are engaged for work that exceeds local capability. Two requirements drive the
tiering:

1. **No remotely-held state.** Conversation and task state lives on the agent VM. No
   provider holds anything the system depends on to resume.
2. **Vendor fluidity.** The frontier tier should be swappable — Anthropic, OpenAI,
   Google, or a vendor that does not exist yet — as a configuration change, not a
   refactor.

These two are only in tension if portability is required *within* a session. Making
frontier engagements **ephemeral and self-contained** (A13) allows vendor-specific
session data to be used freely inside a delegation while keeping the durable record
vendor-neutral.

Requirements:
- **One client interface**, `complete(canonical_log, tools, model) -> canonical_events`,
  present at the **first real LLM call** — not introduced as a later cutover. The same
  reasoning that produced the `Transport` abstraction.
- **LiteLLM in front of everything, from the start.** The agent VM gets exactly one
  LLM destination; LiteLLM fans out to the local inference host and to frontier
  vendors behind it. This also resolves the RFC1918 conflict in B3 — a LAN-resident
  inference host would otherwise need a second hole punched in the egress allowlist.
- **Passthrough for the vendor driving a given session** (`/v1/messages` for
  Anthropic), so vendor-specific features are not lost on the path where they matter —
  delegations (A13) always take this path. **Normalized routing for everything else**;
  the canonical log (A10) reconciles the difference.
- **Fallback chains configured at LiteLLM.** Rate limits and outages differ per
  vendor; a frontier failure should not kill a task.
- **No vendor credentials on the agent VM.** LiteLLM holds every key, sourced from
  Bitwarden machine accounts (A7); the agent holds one LiteLLM token. Adding a vendor
  does not touch the agent VM, and a compromised agent leaks no provider keys.
- Config from env/`.env`, **nothing hardcoded**: base URL, auth token, and the role
  mapping (A11). Changing any of it is an env change + restart, nothing more.
- **Budget caps and spend alerts live on the LiteLLM virtual key**, not in agent code.

### A6. Browser Tool

**Purpose:** drive web UIs. Shared implementation; each agent points it at a
different network with different trust.

Requirements:
- **Playwright** (headless Chromium), exposed to the model as tool actions:
  navigate, click, type, read DOM, screenshot. Screenshot + accessibility-tree loop
  so the model can operate UIs it wasn't trained on.
- Runs **inside the agent's VM**. Network reachability is set by the agent's
  deployment, not the tool.
- Must support authenticated sessions (for the sysadmin agent's admin UIs) while
  being usable without auth (dev agent's test apps).

### A7. Secrets Client

**Purpose:** fetch credentials at runtime instead of baking them into images.

Requirements:
- Client for **Bitwarden/Vaultwarden** via **machine account / Secrets Manager** (or
  `bw` CLI + API key). Not the personal vault.
- Reads only from a **dedicated, minimally-scoped machine account** per agent.
- A single **bootstrap token** (the machine-account credential) is the one secret
  that lives on the box: tight file perms, never in image or repo.
- (Deferred, not first cut: dynamic/short-lived secrets via Vault/OpenBao. Leave the
  interface able to grow toward this; don't build it yet.)

### A8. SSO Identity

**Purpose:** each agent authenticates as itself.

Requirements:
- Each agent is a dedicated **LLDAP** principal (`svc-nisse-dev`, `svc-nisse-ops`) in
  a tightly-scoped **Authelia** group. Never the operator's account.
- Support an **unattended-auth path**: because interactive MFA is impossible for a
  headless agent, allow a narrow **IP-scoped, group-scoped, target-scoped**
  single-factor exception. Keep it specific on all three axes.

### A9. Logging

**Purpose:** observability without bloating state.

Requirements: structured logs to journald / external log stack, **not** kept
in-container. Every gate crossing, tool call, and escalation is logged.

### A10. Canonical Transcript Schema

The durable record is a **canonical event log** in SQLite, not a provider-shaped
`messages` array. Provider request bodies are rendered at call time from this log.

| Event | Payload |
|---|---|
| `user_message` | text, source (Matrix event ID) |
| `assistant_text` | text, model that produced it |
| `tool_call` | canonical call ID, tool name, arguments |
| `tool_result` | canonical call ID, result, success/failure |
| `plan_submitted` | plan body, authoring model |
| `gate_opened` | gate type, reason |
| `delegation_started` | brief, target role, resolved model, caps |
| `delegation_completed` | handback note, artifacts, tokens, duration, exit status |
| `compaction` | summary, producing model, range of events replaced |

Rules:
- **Canonical tool-call IDs are minted locally.** Provider IDs are mapped at the
  boundary and never used as a join key. Providers differ in both ID format and in
  where tool calls sit structurally (content blocks vs. a separate array on the
  assistant message).
- **No provider-side handles for content.** If a Files API is used, store the bytes or
  a content-addressed local path; upload or inline at render time. A provider file ID
  must never be the only reference to something.
- **Vendor-opaque reasoning artifacts are stored, tagged with the producing model, and
  never replayed outside the session that produced them.** This covers Anthropic
  signed thinking blocks, OpenAI encrypted reasoning items, and Gemini thought
  signatures. Inside a delegation, replay per that vendor's rules. On delegation exit,
  the artifacts become archival only. A "session" here is the **delegation**, not the
  process lifetime: a delegation resumed after a crash (A3) is the same session and
  may replay its own artifacts.
- **Token budgeting is a render-time concern.** Context window and tokenizer are
  properties of the model being called. Compute budget when building the request, not
  when writing the log.
- **Compaction summaries are first-class log events** with the producing model
  recorded, so a tier change does not silently inherit opaquely-compacted history.
- **Prompt cache breakpoints are placed by the renderer and never stored.** A tier
  change means a cold cache — a cost event, not a correctness event.

### A11. Role Configuration

Model identity is expressed as **roles**, resolved to concrete model strings in config:

```yaml
roles:
  executor:    local/<model>          # drives the loop, mechanical work
  planner:     <frontier>             # authors plans
  delegate:    <frontier>             # scoped sub-task execution
  consultant:  <frontier>             # advisory, no repo access
```

"Best model this week" is a config edit. **The resolved mapping is pinned at task
start** and recorded in the log, so a mid-task config change cannot split a task
across two planners.

### A12. Escalation Triggers

The local model does **not** decide when it needs help. Small models are poorly
calibrated about their own competence and will escalate either constantly or never.
Triggers are observable from outside the model:

1. **Phase policy.** Planning and architecture escalate unconditionally. Mechanical
   edits stay local.
2. **Failure count.** Local attempts; tests fail twice; escalate with the failures
   attached to the brief.

A model-invoked `ask_expert` (A14) exists in addition to these, never as a
replacement, and carries a per-task budget so a confused executor cannot spend twenty
consults circling.

### A13. Delegation

Escalation is **delegation**: a frontier model receives a scoped sub-task, works
autonomously with full tool access, and hands back. It is not a consult, and the local
model does not interpret or re-derive the frontier model's reasoning.

**Harness — one, not one per vendor.** Do not adopt each vendor's agent SDK to obtain
their best agentic behavior; that recreates the lock-in this design avoids, relocated.
One loop, one cap regime, one handback contract.

**Normalization lives in the A10 renderer, not on the wire.** Vendors disagree
structurally about tool calling (content blocks vs. `tool_calls` arrays, tool-result
placement, streaming deltas) and about reasoning artifacts (signed thinking blocks,
encrypted reasoning items, thought signatures) — and a normalizing middlebox is lossy
exactly where a long-running delegation lives. So:

- **Delegations run over LiteLLM's passthrough route** (`/v1/messages` for Anthropic,
  per A5), with the A10 renderer producing the vendor-native request body. The
  renderer is a per-vendor adapter by definition — it already has to know each wire
  shape to render the canonical log.
- **LiteLLM's normalized route is reserved for calls where fidelity is cheap:**
  `ask_expert` (A14), fallback chains, and trialing a new vendor before its renderer
  adapter exists.
- **Adding a vendor** therefore means writing one renderer adapter — or provisionally
  accepting LiteLLM's normalization for that vendor and validating it against the A16
  eval set before trusting it with delegations. A day-one build needs exactly one
  adapter: the vendor actually in use.

**Scoping and authority.** Delegation sits **inside** the plan gate, not around it. A
frontier sub-agent that acts freely would bypass `submit_plan` (A4), which is the
structural control the design rests on.

- **Planning is itself a delegation** that produces a plan and stops. Human approves.
- **Execution delegations are scoped to a sub-task the approved plan already
  contains.**

The frontier model is smart twice; neither time is it authorized to expand the work.

Because plans may be executed by a materially weaker model, the planner system prompt
must require **explicit, low-inference plans** — the executor cannot reconstruct
intent.

**Isolation and caps.**
- **Branch at delegation start.** Abort discards the branch. This gives delegations the
  same atomicity property that boot-time reconciliation (A3) gives interrupted state
  transitions.
- **Hard caps on turns, tokens, and wall clock.**
- **Checkpoint delegation events as they arrive.** Every `tool_call` / `tool_result` /
  assistant turn inside a delegation is durably written to the canonical log when it
  happens (per A2), never batched at handback. This is what makes a delegation
  resumable after a crash — the deterministic resume rule lives in A3.
- **Defined failure return.** On cap breach or explicit failure, the local loop gates
  to `AWAIT_INPUT` rather than resuming on top of a half-edited tree.

**Handback.** The final delegation action is a **structured handback note written for a
weaker reader**, not a human-facing summary:

- what changed
- what is now true that was not before
- what to do next

That note plus the resulting file state is the entire interface back to the local loop.
The delegation's internal transcript is stored alongside the composite event,
vendor-tagged, and never replayed into anything.

#### Rejected alternative: ask-and-apply

An alternative was considered in which the local model asks a frontier model for advice
and applies the answer itself. **Rejected.**

Its principal advantage was blast radius — the frontier model never touches the repo,
and everything flows through a gated local loop. That was explicitly not a concern
here.

Without it, the comparison is lopsided:

- Delegation lets the frontier model read the code, run the tests, and see the
  failures. Ask-and-apply forces it to reason blind from whatever the local model
  thought to include in the brief, and small models write poor briefs.
- The cost advantage is illusory: one delegation that succeeds beats three consult
  round-trips that do not.
- Any variant where the local model *interprets* rather than applies verbatim
  reintroduces the capability gap downstream of the expensive call.

The diff-application machinery, verbatim-fidelity rules, and local-model-as-verifier
design are all dropped.

### A14. `ask_expert` (retained, minimal)

One narrow case survives delegation: **questions requiring no repo access.** "Which of
these two approaches is better," "why does this library behave this way," "is this API
deprecated."

- Single stateless call, text answer.
- No diff application, no fidelity concerns, no session state.
- Reuses the delegation brief format.
- Structured output contracted **at the prompt level, not the API level** — vendors
  differ on JSON mode and schema enforcement. Parse, validate against a local schema,
  retry once.

Expected implementation size: small. Do not let it grow into a second escalation
subsystem.

### A15. Local Model Accommodations

- **Constrained decoding for tool-call JSON.**
- **Reduced tool schema for the executor role:** fewer tools, flatter arguments, no
  deeply nested optionals. Maintain **two tool manifests keyed on role**, rendered at
  request time like everything else.

### A16. Delegation Evaluation Capture

Capture a **consult/delegation eval set** from day one. Each `delegation_started` /
`delegation_completed` pair (A10) is already a clean input-output record; the
additional cost is marking outcomes as good or bad.

Without this, "best model this week" is decided by benchmark chatter, and swapping the
frontier tier becomes something to avoid because the effect is unmeasurable. With it, a
new model is a config change and a re-run.

---

## PART B — THE DEV AGENT / `nisse-dev` (build first)

A thin app over `nisse-core`. Plans and implements software projects; produces PRs.

### B1. Workflow (state machine)

```
IDLE
  → [use <project>]            clone repo, begin architecture planning
PLANNING_ARCH
  → agent drafts architecture + implementation plan → submit_plan
AWAIT_ARCH_SIGNOFF             ← human gate (approve / reject+feedback)
  →
MILESTONE_PLANNING
  → agent drafts plan for a specific milestone → submit_plan
AWAIT_MILESTONE_SIGNOFF        ← human gate
  → [start milestone <n>]
EXECUTING                       autonomous; no per-command approval
  → may enter AWAIT_INPUT (ask_human) any time it's unsure
PR_OPEN
  → open PR; react to review comments and commits (review loop)
  → on acceptance → next milestone, or project complete
```

### B2. Requirements

- **One active project at a time.** Operator selects it.
- **Operator commands** (over transport): `use <project>`, `approve`,
  `reject <feedback>`, `start milestone <n>`, `status`, plus free text during
  planning (forwarded to the agent as answers).
- **Plan gating** per A4: architecture plan and each milestone plan require sign-off
  before the next phase. Enforced structurally. Plans and approvals are externalized
  to Gitea issue comments (Part H, decided); Matrix and Gitea are both approval
  channels, with revision binding and the uniform `approve` UI per A4.
- **Autonomous execution** within a started milestone: builds, runs, tests without
  per-command approval; asks questions any time via `ask_human`.
- **Gitea integration:**
  - *Outbound:* `git` + Gitea API to push branches and open PRs, via a dedicated
    Gitea **service account** with a scoped token. Record any runnable test service's
    URL/port in the PR description.
  - *Inbound:* react to PR comments, commits, pushes. **Poll** the Gitea API
    (15–30s) rather than take webhooks, to keep the VM outbound-only. Intake is
    structured JSON — never via a chat bridge.
  - *On Gitea unavailability:* the poller backs off (exponential, capped at a few
    minutes) and retries indefinitely, logging the outage. Polling is pure intake —
    an outage delays reactions but never fails a task or forces a state transition.
    Outbound Gitea operations that fail mid-flight are already covered by the
    intent log and boot-time reconciliation (A2/A3).
- **Browser (A6)** scoped to the **internal test network only** — used to exercise
  web apps it built and reproduce bugs. **No SSO needed** for this.

### B3. Deployment & Isolation

- **Dedicated VM** (Proxmox), not a bare container — the agent builds and runs Docker
  containers, so it needs its own real Docker daemon. (Docker-in-a-container via
  socket-mount ≈ host root; via dind = nested-storage pain. The VM boundary is the
  correct trust boundary for unattended arbitrary command execution.)
- **Sealed to start:** egress allow-list (LiteLLM, Gitea, package registries), **no
  ingress**. Egress denial must include **RFC1918 except the specific LiteLLM/Gitea
  IPs** — sealed from the internet must also mean no reach to NAS/Proxmox/UDM.
  - **The LAN-resident inference host is not a second exception.** It sits behind
    LiteLLM (A5), so the allowlist keeps exactly one LLM destination regardless of how
    many models or vendors are in play.
- **No Tailscale** on this VM (it routes around firewall policy). Manage via Proxmox
  console / jump host.
- **Own VLAN from day one** (even a VLAN of one) and test services on **predictable
  ports** (e.g. 18000–18099), so opening access later is a firewall edit, not a
  rebuild. (Sealed for now; may relax later.)
- Test containers on a dedicated **internal Docker network with no route out**.
- Build-daemon storage on a **separate mount** from the state volume, so
  `docker system prune` between milestones can't touch agent state.

---

## PART C — THE SYSADMIN AGENT / `nisse-ops` (follow-on)

A thin app over the extracted `nisse-core`. Operates infrastructure; driven by
tickets; gated on consequences. Built after the Dev Agent works.

### C1. How it differs (not reducible to config)

| Axis | Dev Agent | Sysadmin Agent |
|---|---|---|
| Rollback | revert snapshot / discard branch | none for `zfs destroy` or a firewall lockout |
| Trust model | allow-all in sandbox | tiered **allow / deny / confirm** policy |
| Work shape | long milestones → a PR | short reactive tasks + scheduled checks → live changes |
| Driver | Gitea projects | **Vikunja tickets** |
| Placement | sealed, own VLAN, no Tailscale | Tailscale, credentials, broad reach |

Running both roles in one instance would collapse isolation — build them as separate
deployments over the same library.

### C2. Permission Policy Engine *(the defining addition)*

Tiered policy, evaluated on every action:
- **Read-only telemetry** (metrics, logs, status) — free.
- **Reversible service actions** (restart a container) — allowed.
- **High-consequence actions** (storage, networking, auth) — **confirm-first** over
  transport before executing.

This is the sysadmin analog of the dev agent's plan gate: the human boundary sits at
irreversible / high-blast-radius actions.

Special cases fold into the tiers:
- **Browser actions against admin UIs default to confirm-first.** Changes to
  **network gear (UniFi) are the most conservatively gated action in the system** — a
  bad remote UI click can sever the path used to fix it.
- **Action-triggering tickets are operator-scoped** (C4). Auto-generated tickets
  (e.g. from a monitoring alert) route through confirm-first, never drive unattended.

### C3. Interfaces

- **Browser (A6):** into admin UIs. **Prefer real APIs where they exist** (Proxmox,
  TrueNAS good; UniFi partial); use the browser only for genuinely UI-only gaps —
  it's the least stable interface.
- **SSO (A8):** logs into admin UIs as `svc-nisse-ops`. This account is a high-value
  credential; its blast radius equals its Authelia scope, so scope it minimally.
- **Secrets (A7):** UI/SSO logins and service API tokens from Bitwarden.
- **Event intake:** monitoring feeds the orchestrator as **structured JSON**
  (directly or via Alertmanager), never through a Matrix bridge. The agent posts
  curated, state-aware updates to `#ops` ("Gitea 502ing, restarted, back up").

### C4. Ticket-Driven Control (Vikunja)

The sysadmin analog of "dev agent driven by Gitea."
- **Vikunja ticket = task of record** (durable, has status/queue/history). **Matrix =
  live conversation about it** (questions, confirm-first prompts, progress).
- **Flow:** ticket filed → agent picks up → discuss/confirm over Matrix → update
  status as it progresses → close with result.
- **Mechanics:** Vikunja REST API + webhooks (same two-direction, JSON-only pattern
  as Gitea); dedicated service account, scoped token from Bitwarden. Verify the
  installed version's webhooks fire on comment creation; if not, poll comments as
  B2 polls Gitea.
- **Approvals:** Vikunja is an `ApprovalSource` (A4) for plan gates on its own
  tickets, via `approve rev-N` comments. Live **confirm-first** prompts (C2) stay
  Matrix-primary — polling latency makes a ticket comment a poor interactive
  channel — with the outcome written back to the ticket as the record.
- **Trust:** a driving ticket is an instruction channel into an infrastructure-
  changing agent → action-triggering ticket creation is operator-scoped (C2).

### C5. Deployment & Isolation

- Longer-lived, **stateful VM** (not disposable).
- Has **Tailscale** + scoped credentials to reach infrastructure across both sites.
- Updated **in place** via the container pipeline, **restart gated on an idle flag**.

---

## PART D — NOTES / KNOWLEDGE (Outline)

Shared knowledge store for both agents (sysadmin-primary). Not a message bus — see
note below.

Requirements:
- **Read — runbooks as context.** Procedures written in Outline are fetched before
  acting, constraining the agent toward *your* procedures over improvised ones.
- **Write — durable record.** Agent writes change logs / incident notes / findings
  back to Outline: searchable, off-box, and an extra externalized record that
  survives volume loss.
- Access via Outline **REST API**, dedicated service account, token from Bitwarden.

**Deliberately rejected:** using a shared notes store (Outline *or* an Obsidian vault)
as an **agent-to-agent message bus**. It's unstructured, race-prone (last-write-wins,
no delivery semantics), and a shared read-write store both agents touch would
undermine the dev/sysadmin isolation boundary. Agent coordination, if ever needed,
goes through the **human**, **Matrix**, or a **Vikunja ticket** handed from one agent
to another — or a real message queue if genuine multi-agent async ever arrives.

---

## PART E — SUPPORTING SERVICES (installed)

**Status: all supporting services are installed and running.** What remains for each
is per-agent *configuration* (service accounts, scoped tokens, rooms), created as
each agent is built — not standing up the service itself.

- **Matrix homeserver** (Synapse/Conduit): ✅ installed. *Remaining per-agent:* create
  the two bot identities (`@nisse-dev`, `@nisse-ops`), one room per agent + `#ops`.
- **Gitea:** ✅ installed. *Remaining per-agent:* `nisse-dev` service account + scoped
  token; enable native-JSON webhooks or confirm API polling access.
- **LiteLLM proxy:** ✅ installed, and **on the path from the first LLM call** (A5) —
  it is the agent VM's single LLM destination, providing per-agent virtual keys with
  budget caps, central logging, vendor swap, fallback routing, and the Anthropic
  passthrough. *Remaining per-agent:* mint a virtual key; configure routes for the
  local inference host and each frontier vendor, with keys sourced from Bitwarden.
- **Local inference host:** 🔜 **decided in principle, hardware pending** — a
  4090-class server (Part H). Interim: a Jetson Nano running Ollama validates the
  plumbing, and the `executor` role maps to a Haiku-class frontier model via LiteLLM
  until the server arrives. Everything in A5, A12, and A15 assumes the eventual host
  sits on the LAN behind LiteLLM.
- **Vikunja:** ✅ installed. *Remaining (sysadmin):* service account + scoped token;
  webhook/poll config (Part C4).
- **Outline:** ✅ installed. *Remaining per-agent:* service account + API token
  (Part D).
- **Bitwarden/Vaultwarden:** ✅ installed. *Remaining per-agent:* dedicated
  minimally-scoped machine account + bootstrap token (A7).
- **LLDAP + Authelia:** ✅ installed. *Remaining per-agent:* create `svc-nisse-dev` /
  `svc-nisse-ops` principals, scoped groups, and the narrow unattended-auth exception
  (A8).

---

## PART F — DEPLOYMENT MODEL (both agents)

- **Containers + persistent volumes.** Stateless image; named volumes carry state
  across redeploys (and allow optional persistence between deployments).
- **Image delivery:** Gitea Actions → Gitea OCI registry → Watchtower for the *pull*.
- **Gate the restart, not the pull:** an idle/"safe-to-restart" flag; Watchtower must
  never restart mid-work. *Unplanned* interruptions (reboot/crash) are handled by the
  durability layers (A2/A3), not by restart timing.
- **Survive reboots:** VM + Docker service set to start on boot; `restart:
  unless-stopped`; ZFS snapshots of the state dataset (scheduled + on each approval).
- **Terraform + cloud-init:** deferred. When adopted, `bpg/proxmox` + cloud-init,
  VM definitions as code, clean `destroy`/`apply`; the dev agent becomes a candidate
  for immutable rebuilds.

---

## PART G — BUILD ORDER

1. **State store + durability (A2/A3)** — critical path; everything sits on it.
2. **Transport (A1, Matrix) + control loop/gating (A4)** with a **stub agent** that
   just echoes plans — get the approval loop feeling right before any real LLM.
3. **LiteLLM + LLM client interface (A5).** The interface must exist at the **first
   real LLM call**, not as a later cutover. Mint a virtual key with a budget cap;
   verify prompt caching survives the passthrough.
4. **Canonical log schema and renderer (A10).** Before any transcript accumulates in a
   provider-shaped form that would have to be migrated later.
5. **Local executor loop (A11/A12/A15)** — role resolution pinned at task start, phase
   policy, failure-count escalation, reduced tool manifest, constrained decoding.
6. **Gitea layer** — polling intake, PR creation, comment/commit handling.
7. **Delegation harness + plan-gate integration (A13/A14).** Begin capturing the eval
   set here (A16); the records are free once the composite events exist.
8. **Browser tool (A6)** for `nisse-dev` — test apps on the sealed network.
9. First **genuinely unattended milestone** (budget caps + ZFS rollback earn their keep
   here). Crash-atomic writes and reconcile-on-boot must be in place first.
10. **Extract `nisse-core`**; build the **Sysadmin Agent** (`nisse-ops`): policy
    engine (C2), SSO + scoped browser (A8/A6), secrets (A7), Vikunja control (C4),
    Outline runbooks (D), scheduled checks, alert intake.
    - Under a monorepo, extraction is a **package split** within one tree: move code
      behind a package boundary, no release machinery required.
    - Under three repos, it is a **repo split plus a version-pinning contract**:
      `nisse-core` gets tagged releases, and `nisse-dev` pins a version rather than
      tracking `main`, or every core change becomes a coordinated two-repo bump.
    This is the step the open repo-layout decision (Part H) actually bites on.

---

## PART H — OPEN DECISIONS

- **Inference hardware — resolved in principle, no longer blocking.** Target: a
  **4090-class host (24GB VRAM)** running a **24–32B open model** at 4–5-bit
  quantization (candidates: Devstral Small 24B, Qwen3-32B / Qwen3-30B-A3B,
  GPT-OSS-20B). Models of this class are competent at tool calling and mechanical
  edits, which puts the system in "tiering genuinely load-bearing" territory rather
  than "local model as dispatcher." Consequences:
  - **Context, not weights, is the VRAM pressure.** A 32B at 4-bit leaves roughly
    5GB for KV cache, and agent loops render long contexts. Model size vs. context
    length vs. KV-cache quantization is an empirical trade — tune it, and the
    thresholds that depend on local capability (A12 failure count, A14 consult
    budget, A15 manifest size), at build-order step 5 on the real hardware.
  - **Interim posture:** until the server exists, the `executor` role points at a
    cheap frontier model (Haiku-class) through LiteLLM — a config entry under A11,
    nothing structural, so the build order does not stall. The Jetson Nano validates
    plumbing (Ollama → LiteLLM → agent) only; nothing learned from models it can run
    transfers to threshold tuning.
  - If the 24GB class proves insufficient in practice, additional hardware is the
    fallback — the role indirection (A11) makes that swap invisible to the agents.

The rest resolve when reached, not now.

- **Repository layout:** one monorepo with three packages vs. three separate repos.
  Three repos match the build order and give each artifact its own CI and release
  cadence, at the cost of version-pinning `nisse-core` from Gitea and coordinating
  bumps every time the shared interface moves. A monorepo is less friction while
  `nisse-core` is still being extracted from a working `nisse-dev` and its seams are
  still moving. Splitting later is easy; merging later is not. Decide by step 10.
- **Sysadmin browser access to network gear:** standing vs. per-task hand-off. The
  lock-yourself-out risk argues for per-task.
- **Policy action taxonomy (C2):** the concrete list of which operations are
  allow vs. confirm. Best decided with the agent half-built in front of you.
- **Approved-plan externalization — decided: Gitea-backed from the start.** Plans are
  posted as issue comments on the project repo; the approval comment (author,
  timestamp, revision tag) is the durable off-box record, authenticated by Gitea's
  own auth. The state store keeps only a pointer ("gate 3 satisfied by comment 8841"),
  so boot-time reconciliation (A3) can re-derive gate state from Gitea even after
  state-volume loss — external truth wins (A2). Approval mechanics and revision
  binding are specified in A4.

---

## Repository Layout

```
nisse-core/              # shared library (extracted after nisse-dev works)
  transport/             #   matrix | telegram | slack                (A1)
  state/                 #   persistence, reconcile, migration     (A2/A3)
  control/               #   loop, state machine, gating, ask_human   (A4)
  llm/                   #   client interface, role resolution        (A5)
  log/                   #   canonical event log + renderer          (A10)
  delegation/            #   delegation harness, ask_expert       (C1/C3)
  browser/               #   Playwright tool                          (A6)
  secrets/               #   Bitwarden machine-account client         (A7)
  sso/                   #   LLDAP identity + auth path               (A8)
  logging/               #   structured operational logs              (A9)

nisse-dev/               # app: sealed VM                        (Part B)
  workflow (arch/milestone gating), Gitea PR layer, allow-all sandbox
  browser → internal test network only

nisse-ops/               # app: separate host, Tailscale         (Part C)
  policy engine, tiered confirmation, scheduled checks
  Vikunja ticket intake, Outline runbooks/records
  browser → admin UIs (API-first, UI fallback), confirm-first gating

infra/                   # (future) Terraform + cloud-init       (Part F)
  nisse-dev.tf | nisse-ops.tf | templates/
```

Shown as three sibling trees. Under a monorepo they become three packages under one
root; under three repos, three checkouts. Either way the module boundaries are the
same, which is the point of deciding this late (Part H).

`log/` carries the canonical event log and its renderer (A10); `delegation/` carries
the delegation harness and `ask_expert` (A13/A14).