Commit Graph
4 Commits
Author SHA1 Message Date
dongarandClaude Opus 5 ed2f5dfc7f Choose the LLM route by model, and add a capability matrix
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 58s
The arrival of non-Anthropic models made a latent bug observable: the
delegate's client was hardcoded to the Anthropic passthrough, so
pointing NISSE_ROLE_DELEGATE at gpt-6-astra or a local model would
have sent Anthropic-shaped /v1/messages bodies to something that does
not speak them. A5 promises the frontier tier is swappable "as a
configuration change, not a refactor", and it was not.

select_client now picks the route from the model: the vendor-native
adapter where one exists, LiteLLM's normalization otherwise — which is
exactly what A13 prescribes for a vendor whose renderer adapter has
not been written, pending validation against the A16 eval set.

Also adds the capability matrix Part H asks for. It says the
thresholds that depend on local capability — A12's failure count,
A14's consult budget, A15's manifest size — must be tuned "on the real
hardware", and that needs measurement rather than reputation. The
harness probes each model for the things that actually break: whether
it calls a tool at all, whether the plan it submits has substance
(llama3.2 emitted "{"), whether it can consume its own tool result,
and whether prompt-level JSON holds. Opt in with
NISSE_CAPABILITY_MODELS; it spends real tokens across several models.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 23:20:38 +00:00
dongarandClaude Opus 5 af9bdaac8d Stop CI running two suites at once, and retry rate limits
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m5s
CI failed on the delegation branch with a 429 from the proxy: token
limit 200000, 17399 remaining. Not a defect — the live delegation
tests are token-hungry, and every push to a PR branch fired two
identical suites concurrently, since the push and pull_request events
both matched.

- Push now only triggers on main. A PR branch is already covered by
  the pull_request event, so this halves both the wall time and the
  tokens spent per branch
- A 429 is retried once before being surfaced. Losing a ten-turn
  delegation to a transient limit wastes far more than waiting out
  the window, and retrying forever would hide a real ceiling. The
  sleep is injectable, so the existing 429 test no longer waits 20
  seconds for real

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 00:18:32 +00:00
dongarandClaude Opus 5 cf85648719 Add LLM-backed planning agent (A4/A11/A13) — step 5a
CI / checks (push) Successful in 12s
CI / checks (pull_request) Successful in 11s
- planner.py: LLMPlanningAgent offers exactly two tools, submit_plan
  and ask_human, so gating stays structural at the tool layer; no
  proceed tool exists. System prompt requires explicit, low-inference
  plans because a weaker model executes them (A13)
- Model output is appended to the canonical log as it arrives, before
  being interpreted, so a crash mid-planning leaves a truthful
  transcript
- A bad turn (prose instead of a tool call, malformed arguments, or a
  failed call) retries once, then raises PlanningError; the
  orchestrator gates to AWAIT_INPUT rather than inventing a plan
- Orchestrator: appends every operator message, plan_submitted, and
  gate_opened to the transcript, so re-planning sees rejection
  feedback as conversation rather than out-of-band parameters; pins
  the A11 role mapping at task start
- llm.py: non-JSON or non-object tool arguments now raise LLMError
  instead of escaping as JSONDecodeError — observed from llama3.2,
  which emits truncated argument strings
- Live planning test is opt-in on NISSE_LIVE_PLANNER_MODEL: planning
  escalates unconditionally (A12) and the local model does not hold
  the tool-call contract, so it awaits a frontier route

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 14:47:12 +00:00
dongarandClaude Fable 5 0c2021fced Add LLM client interface with LiteLLM adapter (A5/A11)
CI / checks (pull_request) Successful in 10s
CI / checks (push) Successful in 9s
PR 2 of the steps-3/4 plan.

- llm.py: LLMClient protocol — complete(events, tools, model) ->
  canonical events — present from the first real LLM call.
  LiteLLMClient renders requests via the A10 renderer, targets the
  proxy's OpenAI-compatible route, and maps provider tool-call IDs
  to locally-minted canonical IDs at the boundary
- RoleMap (A11): executor/planner/delegate/consultant resolved from
  env; pinned() returns the mapping to record at task start
- Config strictly from env: NISSE_LLM_BASE_URL, NISSE_LLM_TOKEN;
  envfile.py is a minimal .env loader where real env wins; .env now
  gitignored with a committed .env.example documenting all variables
  (Matrix ones included for the upcoming transport PR)
- Live smoke test against the proxy is integration-marked and skips
  without credentials — CI stays credential-free per operator
  decision; the test runs locally once .env exists
- httpx added as nisse-dev's HTTP client

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-06 20:57:51 +00:00