Enforce the tool-call contract, asking servers only where it works
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 56s

The executor ended a milestone on any reply without a tool call, and
nothing held a model to the offered tools. Measured on the 4070:
tool_choice "required" is enforced by vLLM (local-fast) and Anthropic,
but qwen3-14b (local-coder) under it cannot stop and writes until the
output ceiling, a known model bug. So enforcement is asked for per
model, and the loop holds the contract itself everywhere.

- The catalog reads supports_tool_choice from LiteLLM's model_info;
  BudgetedClient sends tool_choice only to models declared true, as
  "required" on the normalized route and {"type": "any"} on the
  Anthropic passthrough, and only when tools are offered
- The executor gains an explicit finish tool, the only way to end a
  milestone. Finishing with nothing changed is refused once; a prose
  reply gets one nudge, then escalates; a call to a tool not offered
  counts toward failure_limit, like a failed command; calls after
  finish are recorded as not run
- Clients record the model that actually answered. ModelFacts.answered
  recognises a proxy fallback, which once credited Claude's answers to
  a local model that could not call tools at all
- The context probe reports a fallback instead of trusting it; rerun,
  both local models now refuse over-long prompts, as they should
- The capability matrix checks every answer came from the model asked,
  that models call only offered tools, and that a declared
  supports_tool_choice is true
- Spec A15 and the tuning guide say how enforcement is requested,
  declared and backed up

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-09-24 22:54:49 +00:00
1 parent 389b71a5d1
commit b934c46714
16 files changed
+568 -86

No files matched your search

+19 -1
View File
@@ -215,9 +215,15 @@ class BudgetedClient:
max_output_tokens: int = DEFAULT_MAX_OUTPUT_TOKENS,
keep_recent_turns: int = 6,
safety_margin: float = 0.1,
enforces_tool_choice: Callable[[str], bool] = lambda model: False,
) -> None:
self._inner = inner
self._limits_for = limits_for
# Whether a model's server holds it to the offered tools when asked
# (LiteLLM's supports_tool_choice). Asking one that doesn't can be
# worse than not asking: qwen3-14b can't stop, and rambles until the
# output ceiling.
self._enforces_tool_choice = enforces_tool_choice
self._max_output_tokens = max_output_tokens
self._keep_recent_turns = keep_recent_turns
self._safety_margin = safety_margin
@@ -229,6 +235,10 @@ class BudgetedClient:
def last_usage(self) -> dict[str, int]:
return getattr(self._inner, "last_usage", {}) or {}
@property
def last_model(self) -> str | None:
return getattr(self._inner, "last_model", None)
def budget_for(self, model: str) -> Budget:
window, model_max_output = self._limits_for(model)
ceiling = self._max_output_tokens
@@ -252,7 +262,15 @@ class BudgetedClient:
factor = self._factors.get(model, 1.0)
fitted = fit(events, system, tools, budget, factor)
produced = self._inner.complete(
fitted, tools, model, system=system, max_tokens=budget.max_output
fitted,
tools,
model,
system=system,
max_tokens=budget.max_output,
# Every caller that offers tools needs one called: the executor
# finishes with finish, the planner with submit_plan or
# ask_human, a delegation with hand_back.
require_tool_call=bool(tools) and self._enforces_tool_choice(model),
)
reported = int(self.last_usage.get("input", 0))
if reported > 0: