Discover a model's vendor from LiteLLM instead of its name
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 47s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 47s
Route selection sniffed model names: anything starting "claude-" or "anthropic/" took the vendor-native passthrough, everything else took the normalized route. That works only while an alias names its vendor. A semantic role alias like "remote-architecture" is meant to point at Anthropic today and somewhere else tomorrow, and the name says nothing either way -- so it would have taken the normalized route regardless of what was behind it, silently losing the fidelity A13 wants for delegations. LiteLLM already knows the answer. /model/info reports each alias's backing model, provider, and declared tool support, so the agent asks at startup and routes on the provider it discovers. That keeps vendor knowledge where A5 already put it -- in the proxy -- rather than duplicating it as name patterns here. There is deliberately no name-based fallback: PASSTHROUGH_PREFIXES and has_native_adapter are gone, and select_client now requires a catalog. A second heuristic could only ever disagree with the proxy. An unknown model raises instead of guessing a route. Misconfiguration is therefore loud and early. build_agent validates every role against the catalog before it clones anything, naming all the bad ones at once. Discovery is per key -- /model/info returns only what the presented token may use -- so the same check catches a role pointed at a real model this key cannot reach. An unreachable proxy is reported separately, since an outage and a typo call for different responses from the operator. Two things now get recorded rather than inferred: the resolved model is pinned alongside the role mapping at task start, for the reason A11 already pins roles, and DelegationStarted carries backing_model so A16 eval history is not averaged across whatever an alias happened to point at. A model LiteLLM declares tool-less warns at startup rather than refusing; A15 expects weak local models to be accommodated, and they are observed driving tool loops anyway. Verified live: all four aliases the production key can see route correctly through discovery, and each chosen route works. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
231b406393
commit
e38adf40ca
10 files changed
+574
-76
No files matched your search
@@ -12,6 +12,7 @@ from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
|
||||
from nisse_dev.approvals import PlanBoard
|
||||
from nisse_dev.catalog import ModelCatalog, select_client
|
||||
from nisse_dev.config import AgentConfig
|
||||
from nisse_dev.control import Orchestrator
|
||||
from nisse_dev.delegation import Delegation
|
||||
@@ -19,11 +20,7 @@ from nisse_dev.executor import LLMExecutor
|
||||
from nisse_dev.expert import ExpertDesk
|
||||
from nisse_dev.git import GitRepo, authenticated_url
|
||||
from nisse_dev.gitea import Comment, GiteaClient
|
||||
from nisse_dev.llm import (
|
||||
AnthropicPassthroughClient,
|
||||
LiteLLMClient,
|
||||
select_client,
|
||||
)
|
||||
from nisse_dev.llm import AnthropicPassthroughClient, LiteLLMClient
|
||||
from nisse_dev.matrix import MatrixTransport
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.planner import LLMPlanningAgent
|
||||
@@ -32,6 +29,35 @@ from nisse_dev.publish import MilestonePublisher
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import Workspace
|
||||
|
||||
TOOL_DRIVING_ROLES = ("planner", "executor", "delegate")
|
||||
|
||||
|
||||
def discover_models(config: AgentConfig, oplog: OpLog) -> ModelCatalog:
|
||||
"""Resolve every configured role against LiteLLM before anything runs.
|
||||
|
||||
Refusing to boot is the whole point: a role pointed at a model this key
|
||||
cannot see is a misconfiguration, and the cheapest place to learn that is
|
||||
here rather than ten minutes into a milestone. An unreachable proxy is
|
||||
reported separately -- it is an outage, not a mistake, and telling them
|
||||
apart is what lets an operator know whether to edit config or wait.
|
||||
"""
|
||||
catalog = ModelCatalog.discover(config.llm_base_url, config.llm_token)
|
||||
catalog.validate(config.roles.pinned())
|
||||
for role in TOOL_DRIVING_ROLES:
|
||||
facts = catalog.facts(config.roles.resolve(role))
|
||||
if facts.supports_tools is False:
|
||||
# A warning, not a refusal: models LiteLLM declares tool-less are
|
||||
# observed driving tool loops anyway, and A15 expects weak local
|
||||
# models to be accommodated rather than excluded.
|
||||
oplog.emit(
|
||||
"model_tools_unsupported",
|
||||
cause="startup",
|
||||
role=role,
|
||||
model=facts.alias,
|
||||
backing=facts.backing,
|
||||
)
|
||||
return catalog
|
||||
|
||||
|
||||
@dataclass
|
||||
class Agent:
|
||||
@@ -91,6 +117,9 @@ def build_agent(
|
||||
) -> Agent:
|
||||
oplog = OpLog("nisse-dev")
|
||||
store = StateStore(config.state_db, oplog=oplog)
|
||||
# Before the clone, before the token checks: a typo'd role name should
|
||||
# cost a second, not a repository.
|
||||
catalog = discover_models(config, oplog)
|
||||
|
||||
gitea = GiteaClient(config.gitea_url, config.gitea_token)
|
||||
# Whoami rather than another config variable: it names the identity every
|
||||
@@ -113,11 +142,12 @@ def build_agent(
|
||||
expert = ExpertDesk(normalized, store, config.roles)
|
||||
delegate_model = config.roles.resolve("delegate")
|
||||
delegation = Delegation(
|
||||
select_client(delegate_model, normalized, passthrough),
|
||||
select_client(delegate_model, normalized, passthrough, catalog),
|
||||
store,
|
||||
workspace,
|
||||
git_repo,
|
||||
model=delegate_model,
|
||||
backing_model=catalog.facts(delegate_model).backing,
|
||||
)
|
||||
executor = LLMExecutor(
|
||||
normalized,
|
||||
@@ -162,6 +192,7 @@ def build_agent(
|
||||
board=PlanBoard(gitea, config.gitea_repo),
|
||||
delegate=delegation,
|
||||
poller_factory=poller_factory,
|
||||
catalog=catalog,
|
||||
)
|
||||
return Agent(
|
||||
config=config,
|
||||
|
||||
Reference in new issue
Block a user