4 Commits

Include renames

Author	SHA1	Message	Date
dongarandClaude Opus 5.5	58b439088f	Pass the capability matrix's models as a pytest option ... NISSE_CAPABILITY_MODELS was neither a secret nor configuration: which models to measure is a question for one run, so it is now an argument, --capability-models=a,b, declared in tests/conftest.py where pytest reads options at startup. With no models the matrix skips as before, so CI is unchanged. The last non-secret NISSE_* variable other than NISSE_CONFIG and NISSE_ENV is gone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-23 23:32:44 +00:00
dongarandClaude Opus 5.5	15d69799bf	Make the config file the only home for settings ... With no production instance yet, there is no reason to carry two ways of configuring the agent. Every environment variable that duplicated a config key is gone; the environment now supplies only the secrets the file references by ${…}, plus NISSE_CONFIG and NISSE_ENV to find the files. - AgentConfig loads from a file (load, from_file) or a parsed tree (from_tree, for tests); a missing file is an error naming the example - The from_env constructors on LiteLLMClient, AnthropicPassthroughClient, GiteaClient, MatrixConfig and RoleMap are removed, with the env reading for max output tokens; clients take their settings from config - The live integration tests share a committed agent config, tests/integration/nisse-test.yaml, aimed at test resources, and one conftest that loads it and builds clients. NISSE_LIVE_PLANNER_MODEL becomes its planner role; NISSE_CAPABILITY_MODELS stays as a per-run switch - A missing secret now fails the live tests instead of skipping them - CI passes only the three tokens - .env.example is secrets only; spec A5 says config comes from the file Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-23 23:25:58 +00:00
dongarandClaude Opus 5	e38adf40ca	Discover a model's vendor from LiteLLM instead of its name ... Route selection sniffed model names: anything starting "claude-" or "anthropic/" took the vendor-native passthrough, everything else took the normalized route. That works only while an alias names its vendor. A semantic role alias like "remote-architecture" is meant to point at Anthropic today and somewhere else tomorrow, and the name says nothing either way -- so it would have taken the normalized route regardless of what was behind it, silently losing the fidelity A13 wants for delegations. LiteLLM already knows the answer. /model/info reports each alias's backing model, provider, and declared tool support, so the agent asks at startup and routes on the provider it discovers. That keeps vendor knowledge where A5 already put it -- in the proxy -- rather than duplicating it as name patterns here. There is deliberately no name-based fallback: PASSTHROUGH_PREFIXES and has_native_adapter are gone, and select_client now requires a catalog. A second heuristic could only ever disagree with the proxy. An unknown model raises instead of guessing a route. Misconfiguration is therefore loud and early. build_agent validates every role against the catalog before it clones anything, naming all the bad ones at once. Discovery is per key -- /model/info returns only what the presented token may use -- so the same check catches a role pointed at a real model this key cannot reach. An unreachable proxy is reported separately, since an outage and a typo call for different responses from the operator. Two things now get recorded rather than inferred: the resolved model is pinned alongside the role mapping at task start, for the reason A11 already pins roles, and DelegationStarted carries backing_model so A16 eval history is not averaged across whatever an alias happened to point at. A model LiteLLM declares tool-less warns at startup rather than refusing; A15 expects weak local models to be accommodated, and they are observed driving tool loops anyway. Verified live: all four aliases the production key can see route correctly through discovery, and each chosen route works. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-23 01:47:58 +00:00
dongarandClaude Opus 5	ed2f5dfc7f	Choose the LLM route by model, and add a capability matrix ... The arrival of non-Anthropic models made a latent bug observable: the delegate's client was hardcoded to the Anthropic passthrough, so pointing NISSE_ROLE_DELEGATE at gpt-6-astra or a local model would have sent Anthropic-shaped /v1/messages bodies to something that does not speak them. A5 promises the frontier tier is swappable "as a configuration change, not a refactor", and it was not. select_client now picks the route from the model: the vendor-native adapter where one exists, LiteLLM's normalization otherwise — which is exactly what A13 prescribes for a vendor whose renderer adapter has not been written, pending validation against the A16 eval set. Also adds the capability matrix Part H asks for. It says the thresholds that depend on local capability — A12's failure count, A14's consult budget, A15's manifest size — must be tuned "on the real hardware", and that needs measurement rather than reputation. The harness probes each model for the things that actually break: whether it calls a tool at all, whether the plan it submits has substance (llama3.2 emitted "{"), whether it can consume its own tool result, and whether prompt-level JSON holds. Opt in with NISSE_CAPABILITY_MODELS; it spends real tokens across several models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-22 23:20:38 +00:00