diff --git a/docs/Architecture-Build-Spec.md b/docs/Architecture-Build-Spec.md index b795a53..6430cbc 100644 --- a/docs/Architecture-Build-Spec.md +++ b/docs/Architecture-Build-Spec.md @@ -1414,6 +1414,38 @@ The rest resolve when reached, not now. and ticket-driven ops work already require one. Whether every gated work item needs one (for instance, an ops question that turns into a change to make) is best settled once C2's step sequences are in use. +- **Open: sharing the local inference host between agents.** Each agent keeps + one active work item (0.2), but `nisse-dev` and `nisse-ops` are separate agents, + and both send their local roles through LiteLLM to the one 24GB host. llama-swap + there keeps one model resident at a time. If the two agents' local roles map to + different models, interleaved requests reload weights on every switch (10–60 s + each), and each agent waits on the other's loads. LiteLLM's per-key rate limits + don't address this: they cap one key's throughput rather than share the host, + and know nothing about which model is loaded. (Measured 2026-09-28: they only + throttled the benchmark, and the token limiter leaked until LiteLLM restarted.) + - **Preferred: one local model for every local role.** No swaps at all. It + needs one model good enough as the dev executor and for ops' Home Assistant + tool calls. The executor benchmark and the ops evaluation decide whether one + exists. More agents are planned for other jobs, which makes one shared model + more valuable still, and OpenSCAD and video work would add vision (next + item). + - **Fallback: agents tag in and out.** If no single model fits, the agents + coordinate their use of the host rather than interleave: an agent holds the + host for a stretch of work on its model, then hands it over, so a swap happens + per hand-over rather than per request. Two models co-resident (llama-swap + groups), where VRAM allows, can reduce how often that's needed. How the + hand-over works (a lease, a queue, or priority for ops alerts) is only worth + designing if the preferred option fails. +- **Parked: vision.** OpenSCAD development needs it: the language describes 3D + parts, and the only way to check a change is to render it and look. That work + may belong to `nisse-dev` or, being different enough from ordinary code, to an + agent of its own; undecided. Video-file management, planned for a future + agent, needs it too. Ordinary code work doesn't. The local candidates on + the 4090 host already accept images, and video as sampled frames, but the agent + code is text-only. Building it will need tools that return images, image content + in canonical events (A10) and both renderers, storage that keeps the durable log + (A2) small, and image tokens in the context budget. How well a model sees is + measured when that work starts, not before. ---