Discover a model's vendor from LiteLLM instead of its name
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 47s

Route selection sniffed model names: anything starting "claude-" or
"anthropic/" took the vendor-native passthrough, everything else took the
normalized route. That works only while an alias names its vendor. A
semantic role alias like "remote-architecture" is meant to point at
Anthropic today and somewhere else tomorrow, and the name says nothing
either way -- so it would have taken the normalized route regardless of
what was behind it, silently losing the fidelity A13 wants for delegations.

LiteLLM already knows the answer. /model/info reports each alias's backing
model, provider, and declared tool support, so the agent asks at startup and
routes on the provider it discovers. That keeps vendor knowledge where A5
already put it -- in the proxy -- rather than duplicating it as name patterns
here.

There is deliberately no name-based fallback: PASSTHROUGH_PREFIXES and
has_native_adapter are gone, and select_client now requires a catalog. A
second heuristic could only ever disagree with the proxy. An unknown model
raises instead of guessing a route.

Misconfiguration is therefore loud and early. build_agent validates every
role against the catalog before it clones anything, naming all the bad ones
at once. Discovery is per key -- /model/info returns only what the presented
token may use -- so the same check catches a role pointed at a real model
this key cannot reach. An unreachable proxy is reported separately, since an
outage and a typo call for different responses from the operator.

Two things now get recorded rather than inferred: the resolved model is
pinned alongside the role mapping at task start, for the reason A11 already
pins roles, and DelegationStarted carries backing_model so A16 eval history
is not averaged across whatever an alias happened to point at.

A model LiteLLM declares tool-less warns at startup rather than refusing;
A15 expects weak local models to be accommodated, and they are observed
driving tool loops anyway.

Verified live: all four aliases the production key can see route correctly
through discovery, and each chosen route works.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dongarandClaude Opus 5 committed 2026-09-23 01:47:58 +00:00
1 parent 231b406393
commit e38adf40ca
10 files changed
+574 -76

No files matched your search

+37 -6
View File
@@ -12,6 +12,7 @@ from collections.abc import Callable
from dataclasses import dataclass
from nisse_dev.approvals import PlanBoard
from nisse_dev.catalog import ModelCatalog, select_client
from nisse_dev.config import AgentConfig
from nisse_dev.control import Orchestrator
from nisse_dev.delegation import Delegation
@@ -19,11 +20,7 @@ from nisse_dev.executor import LLMExecutor
from nisse_dev.expert import ExpertDesk
from nisse_dev.git import GitRepo, authenticated_url
from nisse_dev.gitea import Comment, GiteaClient
from nisse_dev.llm import (
AnthropicPassthroughClient,
LiteLLMClient,
select_client,
)
from nisse_dev.llm import AnthropicPassthroughClient, LiteLLMClient
from nisse_dev.matrix import MatrixTransport
from nisse_dev.oplog import OpLog
from nisse_dev.planner import LLMPlanningAgent
@@ -32,6 +29,35 @@ from nisse_dev.publish import MilestonePublisher
from nisse_dev.state import StateStore
from nisse_dev.tools import Workspace
TOOL_DRIVING_ROLES = ("planner", "executor", "delegate")
def discover_models(config: AgentConfig, oplog: OpLog) -> ModelCatalog:
"""Resolve every configured role against LiteLLM before anything runs.
Refusing to boot is the whole point: a role pointed at a model this key
cannot see is a misconfiguration, and the cheapest place to learn that is
here rather than ten minutes into a milestone. An unreachable proxy is
reported separately -- it is an outage, not a mistake, and telling them
apart is what lets an operator know whether to edit config or wait.
"""
catalog = ModelCatalog.discover(config.llm_base_url, config.llm_token)
catalog.validate(config.roles.pinned())
for role in TOOL_DRIVING_ROLES:
facts = catalog.facts(config.roles.resolve(role))
if facts.supports_tools is False:
# A warning, not a refusal: models LiteLLM declares tool-less are
# observed driving tool loops anyway, and A15 expects weak local
# models to be accommodated rather than excluded.
oplog.emit(
"model_tools_unsupported",
cause="startup",
role=role,
model=facts.alias,
backing=facts.backing,
)
return catalog
@dataclass
class Agent:
@@ -91,6 +117,9 @@ def build_agent(
) -> Agent:
oplog = OpLog("nisse-dev")
store = StateStore(config.state_db, oplog=oplog)
# Before the clone, before the token checks: a typo'd role name should
# cost a second, not a repository.
catalog = discover_models(config, oplog)
gitea = GiteaClient(config.gitea_url, config.gitea_token)
# Whoami rather than another config variable: it names the identity every
@@ -113,11 +142,12 @@ def build_agent(
expert = ExpertDesk(normalized, store, config.roles)
delegate_model = config.roles.resolve("delegate")
delegation = Delegation(
select_client(delegate_model, normalized, passthrough),
select_client(delegate_model, normalized, passthrough, catalog),
store,
workspace,
git_repo,
model=delegate_model,
backing_model=catalog.facts(delegate_model).backing,
)
executor = LLMExecutor(
normalized,
@@ -162,6 +192,7 @@ def build_agent(
board=PlanBoard(gitea, config.gitea_repo),
delegate=delegation,
poller_factory=poller_factory,
catalog=catalog,
)
return Agent(
config=config,