The client waits RATE_LIMIT_RETRY_S and retries once on a 429, silently.
A benchmark run that waited looked slower than the model is: on
2026-09-27, fast models under the unit-test key's 200k tokens/minute
limit had 20-45 s runs that were probably waits.
Both clients now report last_rate_limit_wait_s after every call, even
one that then fails. The benchmark sums it per run into
rate_limit_wait_s, the report's time column leaves it out, and a new
429 column counts the runs that waited. Results stored before this
load unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Five runs per task for qwen3.8-27b, qwen3.6-35b-a3b, gemma-4-31b,
gemma-4-26b-a4b, gemma-4-12b, mimo-v2.6-9b and granite-4.2-8b, all
served by llama.cpp b11206 through llama-swap. Every one passed 20 of
20. local-fast (vLLM 0.30.0) passed 3 of 20.
Runs were made under a 350 W power cap, recorded in each note, after
the uncapped card throttled for heat. granite-4.2-8b also has one
uncapped set, kept apart by its note.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Planner and delegate use claude-opus-5-5 for the high-end judgement
those roles need. The consultant answers single questions with no tools,
so it uses claude-haiku-4-5. Sonnet has no role.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The 4070's two local models with default tuning, as the baseline the
4090's candidates will be compared with. Every one of the 22 failures
is an escalation from failure_limit: 2; there were no prose turns,
invented tools, context trims, or finished-but-wrong results.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Choosing a local executor model, and tuning its knobs, needs measured
success rates across repeated runs, comparable with every model
measured before.
- Four canned milestones under nisse_dev/bench/tasks: write a function,
fix a bug a failing test exposes, thread an option through several
files, and find one failure in output far past max_output_chars. Each
is an approved plan over a seed project, judged by a hidden check the
model never sees; a reference solution proves each is solvable and
that its check tells the difference
- Runs use the agent's own client wiring, now shared as
make_llm_clients, with delegation and ask_expert off: an escalation is
a failure of the model under test
- Each run is stored as one JSON line in bench/results/, committed, with
the model's backing name and declared facts, a note on how it was
served, the task's content hash, the agent's commit and tuning, the
outcome, turns, misbehaviour counts, context trims, tokens, time and
LiteLLM's cost; a fallback answer can never count as a success
- The report rebuilds the comparison from every stored run, with Wilson
95% intervals, marking runs against older task versions
- scripts/benchmark.py runs and reports; scripts/evaluate_model.sh takes
a new model through the context probe, the capability matrix and the
benchmark in one command
- Clients keep LiteLLM's per-call cost, and BudgetedClient counts the
requests it had to trim
- The tuning guide gains "Benchmarking models"
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The executor ended a milestone on any reply without a tool call, and
nothing held a model to the offered tools. Measured on the 4070:
tool_choice "required" is enforced by vLLM (local-fast) and Anthropic,
but qwen3-14b (local-coder) under it cannot stop and writes until the
output ceiling, a known model bug. So enforcement is asked for per
model, and the loop holds the contract itself everywhere.
- The catalog reads supports_tool_choice from LiteLLM's model_info;
BudgetedClient sends tool_choice only to models declared true, as
"required" on the normalized route and {"type": "any"} on the
Anthropic passthrough, and only when tools are offered
- The executor gains an explicit finish tool, the only way to end a
milestone. Finishing with nothing changed is refused once; a prose
reply gets one nudge, then escalates; a call to a tool not offered
counts toward failure_limit, like a failed command; calls after
finish are recorded as not run
- Clients record the model that actually answered. ModelFacts.answered
recognises a proxy fallback, which once credited Claude's answers to
a local model that could not call tools at all
- The context probe reports a fallback instead of trusting it; rerun,
both local models now refuse over-long prompts, as they should
- The capability matrix checks every answer came from the model asked,
that models call only offered tools, and that a declared
supports_tool_choice is true
- Spec A15 and the tuning guide say how enforcement is requested,
declared and backed up
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every turn re-sent the whole transcript with nothing checking it against
the model's window. On a local model that fails silently: Ollama drops
the start of an over-long prompt, which is where the system prompt and
the plan are. Measured on the Jetson's llama3.2, cut at 4096 tokens with
no error.
- The model catalog reads max_input_tokens and max_output_tokens from
LiteLLM's model_info, and startup refuses a role on a model with no
declared window
- BudgetedClient wraps both routes. It fits the canonical events before
rendering: older tool results cut to head and tail, then the oldest
turns dropped with a note, keeping calls with their results; a request
that still cannot fit fails with ContextOverflow and is never sent.
The output ceiling is the smaller of config and the model's limit
- Tokens are estimated from characters, pessimistically, and the
estimate is corrected from reported prompt sizes, only ever upward
- The executor takes its scope from the log: the latest plan, which is
the approved milestone plan, goes in its system prompt, and it sees
only events after it
- Tool output keeps its head and, mostly, its tail, so a test runner's
failure summary survives truncation
- tuning.context.keep_recent_turns and safety_margin are new knobs
- scripts/probe_context.py checks a declared window against what the
server does; it confirmed llama3.2 cuts at the declared 4096
- The tuning guide covers the budget, declaring a local model's limits,
and the probe
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
NISSE_CAPABILITY_MODELS was neither a secret nor configuration: which
models to measure is a question for one run, so it is now an argument,
--capability-models=a,b, declared in tests/conftest.py where pytest
reads options at startup. With no models the matrix skips as before, so
CI is unchanged. The last non-secret NISSE_* variable other than
NISSE_CONFIG and NISSE_ENV is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With no production instance yet, there is no reason to carry two ways
of configuring the agent. Every environment variable that duplicated a
config key is gone; the environment now supplies only the secrets the
file references by ${…}, plus NISSE_CONFIG and NISSE_ENV to find the
files.
- AgentConfig loads from a file (load, from_file) or a parsed tree
(from_tree, for tests); a missing file is an error naming the example
- The from_env constructors on LiteLLMClient, AnthropicPassthroughClient,
GiteaClient, MatrixConfig and RoleMap are removed, with the env
reading for max output tokens; clients take their settings from config
- The live integration tests share a committed agent config,
tests/integration/nisse-test.yaml, aimed at test resources, and one
conftest that loads it and builds clients. NISSE_LIVE_PLANNER_MODEL
becomes its planner role; NISSE_CAPABILITY_MODELS stays as a per-run
switch
- A missing secret now fails the live tests instead of skipping them
- CI passes only the three tokens
- .env.example is secrets only; spec A5 says config comes from the file
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The thresholds Part H says to tune on the real inference hardware were
hardcoded, so every adjustment would have been a PR. They now live in a
`tuning:` section of a YAML config file, which also replaces flat
environment variables as the home for structure (#32).
- The file (NISSE_CONFIG, default ./nisse-dev.yaml) carries structure
and holds no secrets: credentials are ${VAR} references resolved from
the environment, and a reference to an unset variable is an error
- The environment still wins: every existing NISSE_* variable overrides
its key, and the file is optional, so .env and CI work unchanged
- Validation stays fail-fast and reports every problem at once; unknown
keys are errors, so a misspelt knob fails at startup instead of
silently doing nothing
- The config file joins the state store and .env in the check that
refuses to start when they sit inside the workspace clone
- Tuning covers the executor's turn and failure limits, the consult
budget, the delegation caps, the command timeout and the tool output
limit, which Workspace now takes as a setting; the defaults are the
values the agent already ran with
- llm.max_output_tokens reaches both LLM clients from the config
docs/Tuning-Guide.md explains the knobs for someone learning local
model tuning: how weights, quantization and the KV cache spend VRAM,
what each knob does and what goes wrong at either extreme, and an order
to tune in. nisse-dev.example.yaml documents every key and is checked
by a test so it cannot drift.
Closes#32
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`stop` could not have worked by adding the command alone. Planning
turns, milestones and review responses ran synchronously on the thread
that delivered the message, so the Matrix sync loop read nothing until
the work finished. The state store was also opened on the main thread
and used from the transport and poller threads, which SQLite refuses
with its default same-thread check; the in-memory test transport never
showed it.
A single worker thread now runs every handler and owns the store:
- The worker opens the store, so the same-thread check stays on and
fails loudly if anything touches the store from elsewhere
- Matrix messages are handed to the worker; pollers hand over and wait,
so their cursor never passes an unhandled comment
- A message that arrives during long work gets an immediate "busy; say
'stop' to halt" reply and is recorded, not applied later in a state
the operator was not answering
- Tests keep running inline through InlineWorker, which defers a job
submitted from inside another as the real worker would queue it
`stop` sets a shared Cancel flag straight from the transport's thread.
The executor and delegation loops check it before each model call and
between tool calls; the planner checks it before each turn. Calls left
unrun get a result saying so, so the transcript still renders. A
stopped delegation is discarded like a failed one. The orchestrator
moves to AWAIT_INPUT and asks why the work was stopped. The reply is
recorded for the model, and a stopped planning turn resumes with it as
the answer. `stop` with nothing running says so and changes nothing; at
a gate it is no longer taken as rejection feedback.
Closes#40
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Switching as soon as the active item posts a plan or question means the
operator sees the request and the switch notice seconds apart. A prompt
approval then waits until the other item waits or closes, which can be
a whole milestone. A configurable grace period (default 10 minutes)
lets a watching operator answer before the agent moves on, and still
switches when they are away.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
0.2 previously allowed an autonomous switch only while the active item
waits on the operator. Ready work should not sit behind an idle agent
either, so closing an item (completed or abandoned) is now the second
case. This also defines "ready": the agent can act on the item without
the operator, because it has not started or because intake arrived
while it was parked that answers what it was waiting for. A9's
work_item.switch allows a closed or null `from`, and B2 no longer says
only the operator selects the active project.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Letting the model choose which ready item to switch to adds a model
call that belongs to no work item. Duty runs before a finding already
had the same gap. A10 now defines an agent-level scope for these calls,
kept apart from work item transcripts in both directions. The switch
call sees only code-rendered summaries of the ready items, runs as the
executor, and falls back to the oldest item if it fails. A9 gains a
work_item.switch event, and A18 accepts that an injected alert can at
worst reorder work.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
C2's per-tool allow/deny/confirm taxonomy would have to be written in
advance for a tool catalog that will grow large, and every mistake in it
is friction or a hole. An ops write plan is now a list of literal steps
(tool, target, arguments, optional precondition and expected outcome,
retry allowance), and one approval covers exactly that sequence:
- Code renders the approval message from the steps, flagging
irreversible steps, protected targets, and steps that take down the
agent's own host
- A matcher checks every call: read-only is free, an elevated call must
exactly match the next approved step (arguments normalized against the
tool's schema), and anything else halts to AWAIT_INPUT
- Approvals expire if the plan does not start in time
- Classification shrinks to a versioned policy file: instance privilege,
protected targets, and each elevated tool's target argument, all
failing safe; MCP annotations may only tighten
- Bounded matching and standing approvals are deferred, with reasons
Supporting changes:
- A3: an approved sequence resumes after the agent's own restart only on
a verified expected outcome for the step in flight
- A4: an operator kill switch, `stop`, for every agent
- 0.2: the agent may switch items on its own while the active one waits
on the operator; parking unmounts elevated instances; a Matrix reply
to a parked item binds to that item
- A17: step preconditions replace the staleness bounds, the log-checked
verification precondition and the boot-time re-fetch
- A18 (new): prompt injection, designed around limiting what a
compromised model can do rather than detecting injection
- Part H records the decision and parks the deferred items
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
"Project" was carrying several things at once: the repo, the effort
through arch plan and milestones, and the transcript's scope. The
Sysadmin Agent needs the same distinctions in a different shape, with
standing duties that produce many efforts over one scope. New section
0.2 names them:
- Duty: a standing responsibility that produces work items, and owns
deduplication of its findings
- Work item: the unit of state, identified by a locally minted id
(A9's task_id), optionally linked to an external record
- Scope: what a work item operates on, shared across work items
- Park-and-switch: one active work item, switching only at safe
points, nothing lost while parked; the schema does not assume one
0.1 records this as a bounded exception to extract-after-it-works:
identity is designed ahead, behaviour is not. A2, A4, A9, A10, B1/B2
and C4 now speak in these terms; the transcript is scoped per work
item, and a milestone no longer mints a task_id. Part H parks
concurrency with its costs, and leaves open when a work item must
have an external record.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Part F is restructured around rules that now apply to every agent:
- F1: every agent gets its own VM, for inter-agent isolation and
per-agent credential attribution
- F2: a declared, per-instance default-deny egress allowlist that
covers MCP servers' reach, with the tailnet's 100.64.0.0/10 called
out
- F3: MCP servers per agent VM and per (service, site, privilege),
plus the rejected routed-meta-tools alternative
- F4: the existing image pipeline, with third-party MCP images
digest-pinned and excluded from Watchtower
Consequences land where they belong: site identifiers in Part 0, a
machine account per MCP server instance in A7, B3 and C5 recast as
instances of F1/F2, plan approval required for every write in C2, MCP
as the primary infrastructure path in C3, the network-gear browser
item narrowed to fallback access, and stateful-server relocation
parked in Part H.
The standalone amendment is removed now that its content lives here,
as with the Tiered Inference amendment; it remains in the previous
commit.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Records a design session on MCP server deployment as a standalone
amendment, ahead of folding it into the spec:
- Every agent gets its own VM, justified by inter-agent isolation and
per-agent credential attribution; Docker-in-Docker stays as an extra
Dev Agent reason rather than the rule's foundation
- Egress is a declared, per-agent-instance default-deny allowlist,
covering what MCP servers reach on the agent's behalf, with the
tailnet's 100.64.0.0/10 called out as outside RFC1918
- MCP servers run per agent VM, one instance per (service, site,
privilege), with site in the server identity rather than a tool
argument
- Third-party MCP images are digest-pinned and excluded from Watchtower
- Routed meta-tools rejected; stateful-server relocation parked
Two consequences settled with the operator while drafting: each MCP
server instance gets its own Vaultwarden machine account (A7), and any
write, reversible or not, requires an approved plan (C2).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Route selection sniffed model names: anything starting "claude-" or
"anthropic/" took the vendor-native passthrough, everything else took the
normalized route. That works only while an alias names its vendor. A
semantic role alias like "remote-architecture" is meant to point at
Anthropic today and somewhere else tomorrow, and the name says nothing
either way -- so it would have taken the normalized route regardless of
what was behind it, silently losing the fidelity A13 wants for delegations.
LiteLLM already knows the answer. /model/info reports each alias's backing
model, provider, and declared tool support, so the agent asks at startup and
routes on the provider it discovers. That keeps vendor knowledge where A5
already put it -- in the proxy -- rather than duplicating it as name patterns
here.
There is deliberately no name-based fallback: PASSTHROUGH_PREFIXES and
has_native_adapter are gone, and select_client now requires a catalog. A
second heuristic could only ever disagree with the proxy. An unknown model
raises instead of guessing a route.
Misconfiguration is therefore loud and early. build_agent validates every
role against the catalog before it clones anything, naming all the bad ones
at once. Discovery is per key -- /model/info returns only what the presented
token may use -- so the same check catches a role pointed at a real model
this key cannot reach. An unreachable proxy is reported separately, since an
outage and a typo call for different responses from the operator.
Two things now get recorded rather than inferred: the resolved model is
pinned alongside the role mapping at task start, for the reason A11 already
pins roles, and DelegationStarted carries backing_model so A16 eval history
is not averaged across whatever an alias happened to point at.
A model LiteLLM declares tool-less warns at startup rather than refusing;
A15 expects weak local models to be accommodated, and they are observed
driving tool loops anyway.
Verified live: all four aliases the production key can see route correctly
through discovery, and each chosen route works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An exhausted budget was a special case of something more general: the
work is not at fault. A network blip, a timeout, a 5xx from the proxy,
or a rate limit that did not clear all leave a delegation branch that
may be perfectly good, and discarding it would throw away work for a
reason that has nothing to do with the work.
UpstreamUnavailable now names that family, with BudgetExhausted a
member of it — distinguishable where the difference matters, since a
budget will not clear within the hour while a network blip will. A
delegation stopped by any of them is "interrupted": branch kept,
intent left pending, and A3's resume rule picks it up.
The benefit of the doubt stops at the server's own door. A 4xx that is
not a rate limit means the request itself was wrong, and preserving
branches on the strength of one would hide a real bug — so those still
fail and clean up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Exceeding a LiteLLM spend limit takes external models away for the
rest of the day, which makes three existing gaps consequential rather
than untidy.
- A13 calls the delegation caps hard on "turns, tokens, and wall
clock", and the module docstring repeated it, but the token cap was
decoration: nothing counted, and result.tokens was always zero. It
is now metered from the usage the provider reports, so a confused
delegate cannot spend a day's allowance. A5 keeps the *money* cap on
the LiteLLM key, which is a different thing; this is a scope cap
- The normalized route sent no ceiling at all, so truncation detection
there had nothing to detect against. Both routes now send one
- A budget refusal is recognised distinctly. It is not a rate limit:
retrying cannot help, so the planner no longer burns a second call
to fail identically
And a delegation cut off by an exhausted budget is now "interrupted"
rather than "failed", which preserves its branch and leaves its intent
pending. The work on that branch may be perfectly good; discarding it
because the budget ran out would throw away work for a reason that has
nothing to do with the work. A3's resume rule already knows how to
pick such a delegation back up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measuring Opus found a bug that had been silent all along: nothing
checked stop_reason or finish_reason, so a response cut off mid-answer
was parsed as a complete one.
The symptom was a submit_plan tool call whose required `content` field
was simply absent — structurally valid, substantively empty, and
indistinguishable from a model that chose to send nothing. The cause
was our max_tokens of 4096: Opus writing an architecture plan
routinely spends 3,000-4,000 output tokens, and was observed at 4,064
against that ceiling. Roughly one run in three crossed it.
Worse than the empty plan is what the same failure does elsewhere: a
truncated write_file writes half a file and reports success.
- Both routes now raise TruncatedResponse when the model was cut off.
It subclasses LLMError, so the planner's retry and the executor's
and delegation's gating already handle it without learning a new
type: a cut-off turn is a failed turn
- The default ceiling is 16384 and configurable via
NISSE_MAX_OUTPUT_TOKENS. Output is billed by what is produced, not
by the ceiling, so a low limit buys nothing and costs silent
truncation
Verified: three consecutive Opus planning turns now return plans of
5,700-6,200 characters where one in three previously returned nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The arrival of non-Anthropic models made a latent bug observable: the
delegate's client was hardcoded to the Anthropic passthrough, so
pointing NISSE_ROLE_DELEGATE at gpt-6-astra or a local model would
have sent Anthropic-shaped /v1/messages bodies to something that does
not speak them. A5 promises the frontier tier is swappable "as a
configuration change, not a refactor", and it was not.
select_client now picks the route from the model: the vendor-native
adapter where one exists, LiteLLM's normalization otherwise — which is
exactly what A13 prescribes for a vendor whose renderer adapter has
not been written, pending validation against the A16 eval set.
Also adds the capability matrix Part H asks for. It says the
thresholds that depend on local capability — A12's failure count,
A14's consult budget, A15's manifest size — must be tuned "on the real
hardware", and that needs measurement rather than reputation. The
harness probes each model for the things that actually break: whether
it calls a tool at all, whether the plan it submits has substance
(llama3.2 emitted "{"), whether it can consume its own tool result,
and whether prompt-level JSON holds. Opt in with
NISSE_CAPABILITY_MODELS; it spends real tokens across several models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Answering "is the workspace where the agent runs, or what it works
on?" — it is what it works on, and conflating the two is dangerous
rather than merely untidy.
The agent commits its workspace with git add -A and pushes it, so
anything inside the clone reaches the project repository. A .env
placed there would publish the LiteLLM, Matrix and Gitea tokens as a
side effect of the agent working correctly: silently, once, and
probably unnoticed. Startup now refuses both that and a state store
inside the clone, and .env.example documents the intended layout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing had ever assembled the agent. main.py was still the hello-world
placeholder from the scaffold, and no code outside the test suite
constructed an Orchestrator — which is how two fully built, fully
tested components came to be reachable from nothing: the delegation
harness, and the Gitea comment poller. Unit tests that construct a
component directly cannot catch that; only assembly can.
- config.py validates every variable before anything is built and
names all of them at once, rather than failing at first use twenty
minutes into a milestone on the call that mattered. Whitespace-only
counts as missing, since a token of spaces passes a truthiness check
and fails at the first request instead
- agent.py builds the whole graph and owns startup and shutdown order:
reconcile before accepting commands, and on stop, stop watching,
then stop talking, then close the store last
- The Gitea account is read via whoami rather than configured: it names
the identity every permission is evaluated against, and proves the
token works before anything depends on it
- Poller lifecycle is now a state-machine concern: watch the plan issue
while a gate stands, the PR while it is open, and stop on leaving.
The orchestrator owns when to watch; an injected factory owns how to
build one
- The executor reports each turn, because it runs synchronously inside
the transport callback: without it the agent is silent for a whole
milestone and a working run looks exactly like a hung one
- main.py loads .env, builds, and shuts down cleanly on SIGINT/SIGTERM
The entry-point tests changed shape: nisse-dev no longer greets, it
refuses to start unconfigured and says everything that is missing. Both
tests now run with a deliberately empty environment and a .env path
that does not exist, because the working directory's .env holds live
credentials and a test must never be one cwd away from starting a real
agent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two gaps found by auditing spec against code, the second created by
7b landing the harness without its boot-time half.
**The harness was not connected to anything.** Only a live test
constructed a Delegation; A12's failure-count trigger still gated to
the human, which A13 describes as the fallback rather than the path.
Consecutive command failures now hand the sub-task to the delegate,
with the failures in the brief so it need not rediscover what the
local model learned the hard way. A usable handback lets the local
loop carry on; anything else escalates, because a cap breach leaves a
half-edited tree.
**A3 step 5 was unimplemented.** A crash mid-delegation failed safe —
boot gated on "restarted mid-EXECUTING" — but always gated, even on
the clean cut the spec says should resume, and left the branch orphaned
because _abandon only runs while the process lives.
- A delegation now records an intent before the branch exists, and
settles it whichever way the delegation ends; an aborted delegation
is a finished one, and leaving it pending would make the next boot
believe work was still in flight
- delegation_verdict implements the clean-cut test as an observable
check, never model self-assessment: settled if a completion follows
the start, absent if the branch was never created, gate on a
tool_call with no recorded result, resume otherwise
- recover() sweeps at boot, announcing a resume with a notice rather
than a question, since a clean cut needs no decision from the
operator, and leaving an unclean one pending so the generic sweep
gates on it
- resume() continues the same delegation rather than starting a
second: no new branch, no second delegation_started, because A10
defines a session as the delegation, not the process lifetime
Honest limit, stated in the code: "the branch sits at the expected
commit" is approximated by the branch existing and carrying the work,
since per-turn shas are not recorded in the log, so a branch rewritten
underneath us would go undetected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes step 7.
- expert.py: one stateless call for a question needing no repo access.
Structured output is contracted at the prompt level, since vendors
differ on JSON enforcement, so the reply is parsed and validated
locally with one retry — tolerating a fenced code block, because
models wrap JSON habitually, but not missing fields
- The per-task budget is counted from the canonical log rather than
memory, so it survives a restart and cannot hand a circling executor
a fresh allowance. Exhaustion raises rather than answering: a caller
out of consults should get on with the work or escalate
- Consults are recorded as delegation_started/completed with role
consultant. A16 asks for a "consult/delegation eval set", and
reusing the composite events gets consult evaluation for free rather
than inventing a parallel record
- evals.py pairs starts with completions into eval records, with
verdicts kept outside the canonical log: the log records what
happened, a verdict is a later judgement about it, and judgements
get revised. Schema v3 adds the outcomes table, so a v1 store now
traverses two migrations — a better exercise of the hook than one
- ask_expert is offered to the executor as a tool only when a desk is
configured, in addition to the observable escalation triggers and
never instead of them (A12). Consulting is not mutating, so a
completion claim backed only by consults is still challenged
Found live: passing an empty message list and putting the question in
the system prompt fails outright — the Messages API requires at least
one non-system message. The question now travels as a synthesised user
message that is never appended to the log, which keeps the consult
stateless while remaining a valid request.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Observed rather than assumed, after checking the running instance:
collections carry per-user permissions in three tiers and permissions
are collection-scoped, not per-document. So one collection per writing
principal, with read membership for everyone else.
- States precisely what that enforces: no other agent can write a
given agent's world model. It does not enforce operator-versus-agent
exclusion, since the operator holds manager on their own wiki; that
half is procedural, via the break-glass routine that stops the
agents first. Two different strengths, so the spec no longer implies
one guarantee covers both
- Per-user grants, not groups: Outline groups are not federated with
LLDAP, so using them would put membership somewhere nothing syncs.
Three principals make the manual mapping trivial
- Outline API keys are per-user, so each agent's token must belong to
that agent's own Outline user. A token minted from an admin account
voids every boundary above while appearing to work — the same shape
as the Gitea permission surprise
- Collections identified by ID from config, documents by ID recorded
in the state store, which also makes writes idempotent across a
retry
- The conditional-write requirement softens to a revision *read*:
break-glass stops the agents, so compare-and-swap is unnecessary and
only divergence detection is needed for A3
Also records Outline's MCP server as considered and deferred. A17's
enforcement depends on this library sitting between the model and the
store; MCP's purpose is direct model access, and a procedure the model
may decline to fetch does not constrain it. Still a candidate for the
discretionary search tool, with two things to verify first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two loose ends in the A17 amendment, both found by asking whether the
build order needed changing:
- A17 requires retrieved content to be snapshotted into the canonical
log, but A10's event table — the authoritative list — had no row for
it, so an implementer reading A10 would never know to add one.
knowledge_retrieved now carries the text as read, the document ID,
its revision, and when it was read
- Part G step 10 already scheduled Outline, so the ordering does not
change; it now names A17 rather than only Part D, and says which
half arrives first — the world model, with operator procedures
waiting on the C2 taxonomy they bind to
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Part D said what Outline is for but never how to build against it —
the only external integration with no A-series component, while
Transport, LLM, Browser, Secrets, SSO and Logging all have one. A17
supplies the contract.
Settled here:
- Sharing is read access, never write access. Every document has
exactly one writer, so concurrent writers never exist and Part D's
last-write-wins objection disposes of itself. Default shared
membership is operator + nisse-ops; nisse-dev gets private space
only, since B3 seals it from the LAN and a shared network map is a
map of what it is sealed from
- Agent-owned knowledge is always a cache, never authority: rely on
it to know what exists and where to look, verify a specific fact
against the live system before acting on it
- Enforcement is structural, because a prompt is not enforcement. A
model cannot be stopped from believing what it reads, so the
enforceable goal is narrower — an unverified belief must not become
an irreversible action — and four mechanisms carry it, leaning
hardest on surfacing provenance at the human gate that C2 already
requires
- Never an instruction channel and never an approval channel, stated
both here and in A4 where approvals are specified
- Corrections flow through transport rather than direct edits, which
is better provenance: a hand edit is invisible to the agent's
record, an instruction is authenticated, logged, and reconcilable.
Break-glass reuses Part F's idle flag
- Retrieved content is snapshotted into the canonical log, since A10
forbids a mutable provider handle being the only reference — with
the A3 consequence that boot must re-fetch snapshots for resumed
tasks
- Outage posture is asymmetric: write failures never block, but a
failed read of a procedure bound to a confirm-first action gates,
because proceeding is acting unconstrained on exactly the operation
someone wrote a procedure for
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dongarandClaude Opus 5.5
dev-agent