An exhausted budget was a special case of something more general: the
work is not at fault. A network blip, a timeout, a 5xx from the proxy,
or a rate limit that did not clear all leave a delegation branch that
may be perfectly good, and discarding it would throw away work for a
reason that has nothing to do with the work.
UpstreamUnavailable now names that family, with BudgetExhausted a
member of it — distinguishable where the difference matters, since a
budget will not clear within the hour while a network blip will. A
delegation stopped by any of them is "interrupted": branch kept,
intent left pending, and A3's resume rule picks it up.
The benefit of the doubt stops at the server's own door. A 4xx that is
not a rate limit means the request itself was wrong, and preserving
branches on the strength of one would hide a real bug — so those still
fail and clean up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measuring Opus found a bug that had been silent all along: nothing
checked stop_reason or finish_reason, so a response cut off mid-answer
was parsed as a complete one.
The symptom was a submit_plan tool call whose required `content` field
was simply absent — structurally valid, substantively empty, and
indistinguishable from a model that chose to send nothing. The cause
was our max_tokens of 4096: Opus writing an architecture plan
routinely spends 3,000-4,000 output tokens, and was observed at 4,064
against that ceiling. Roughly one run in three crossed it.
Worse than the empty plan is what the same failure does elsewhere: a
truncated write_file writes half a file and reports success.
- Both routes now raise TruncatedResponse when the model was cut off.
It subclasses LLMError, so the planner's retry and the executor's
and delegation's gating already handle it without learning a new
type: a cut-off turn is a failed turn
- The default ceiling is 16384 and configurable via
NISSE_MAX_OUTPUT_TOKENS. Output is billed by what is produced, not
by the ceiling, so a low limit buys nothing and costs silent
truncation
Verified: three consecutive Opus planning turns now return plans of
5,700-6,200 characters where one in three previously returned nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The arrival of non-Anthropic models made a latent bug observable: the
delegate's client was hardcoded to the Anthropic passthrough, so
pointing NISSE_ROLE_DELEGATE at gpt-6-astra or a local model would
have sent Anthropic-shaped /v1/messages bodies to something that does
not speak them. A5 promises the frontier tier is swappable "as a
configuration change, not a refactor", and it was not.
select_client now picks the route from the model: the vendor-native
adapter where one exists, LiteLLM's normalization otherwise — which is
exactly what A13 prescribes for a vendor whose renderer adapter has
not been written, pending validation against the A16 eval set.
Also adds the capability matrix Part H asks for. It says the
thresholds that depend on local capability — A12's failure count,
A14's consult budget, A15's manifest size — must be tuned "on the real
hardware", and that needs measurement rather than reputation. The
harness probes each model for the things that actually break: whether
it calls a tool at all, whether the plan it submits has substance
(llama3.2 emitted "{"), whether it can consume its own tool result,
and whether prompt-level JSON holds. Opt in with
NISSE_CAPABILITY_MODELS; it spends real tokens across several models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI failed on the delegation branch with a 429 from the proxy: token
limit 200000, 17399 remaining. Not a defect — the live delegation
tests are token-hungry, and every push to a PR branch fired two
identical suites concurrently, since the push and pull_request events
both matched.
- Push now only triggers on main. A PR branch is already covered by
the pull_request event, so this halves both the wall time and the
tokens spent per branch
- A 429 is retried once before being surfaced. Losing a ten-turn
delegation to a transient limit wastes far more than waiting out
the window, and retrying forever would hide a real ceiling. The
sleep is injectable, so the existing 429 test no longer waits 20
seconds for real
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- planner.py: LLMPlanningAgent offers exactly two tools, submit_plan
and ask_human, so gating stays structural at the tool layer; no
proceed tool exists. System prompt requires explicit, low-inference
plans because a weaker model executes them (A13)
- Model output is appended to the canonical log as it arrives, before
being interpreted, so a crash mid-planning leaves a truthful
transcript
- A bad turn (prose instead of a tool call, malformed arguments, or a
failed call) retries once, then raises PlanningError; the
orchestrator gates to AWAIT_INPUT rather than inventing a plan
- Orchestrator: appends every operator message, plan_submitted, and
gate_opened to the transcript, so re-planning sees rejection
feedback as conversation rather than out-of-band parameters; pins
the A11 role mapping at task start
- llm.py: non-JSON or non-object tool arguments now raise LLMError
instead of escaping as JSONDecodeError — observed from llama3.2,
which emits truncated argument strings
- Live planning test is opt-in on NISSE_LIVE_PLANNER_MODEL: planning
escalates unconditionally (A12) and the local model does not hold
the tool-call contract, so it awaits a frontier route
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR 2 of the steps-3/4 plan.
- llm.py: LLMClient protocol — complete(events, tools, model) ->
canonical events — present from the first real LLM call.
LiteLLMClient renders requests via the A10 renderer, targets the
proxy's OpenAI-compatible route, and maps provider tool-call IDs
to locally-minted canonical IDs at the boundary
- RoleMap (A11): executor/planner/delegate/consultant resolved from
env; pinned() returns the mapping to record at task start
- Config strictly from env: NISSE_LLM_BASE_URL, NISSE_LLM_TOKEN;
envfile.py is a minimal .env loader where real env wins; .env now
gitignored with a committed .env.example documenting all variables
(Matrix ones included for the upcoming transport PR)
- Live smoke test against the proxy is integration-marked and skips
without credentials — CI stays credential-free per operator
decision; the test runs locally once .env exists
- httpx added as nisse-dev's HTTP client
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dongarandClaude Opus 5