Enforce the tool-call contract, asking servers only where it works

The executor ended a milestone on any reply without a tool call, and
nothing held a model to the offered tools. Measured on the 4070:
tool_choice "required" is enforced by vLLM (local-fast) and Anthropic,
but qwen3-14b (local-coder) under it cannot stop and writes until the
output ceiling, a known model bug. So enforcement is asked for per
model, and the loop holds the contract itself everywhere.

- The catalog reads supports_tool_choice from LiteLLM's model_info;
  BudgetedClient sends tool_choice only to models declared true, as
  "required" on the normalized route and {"type": "any"} on the
  Anthropic passthrough, and only when tools are offered
- The executor gains an explicit finish tool, the only way to end a
  milestone. Finishing with nothing changed is refused once; a prose
  reply gets one nudge, then escalates; a call to a tool not offered
  counts toward failure_limit, like a failed command; calls after
  finish are recorded as not run
- Clients record the model that actually answered. ModelFacts.answered
  recognises a proxy fallback, which once credited Claude's answers to
  a local model that could not call tools at all
- The context probe reports a fallback instead of trusting it; rerun,
  both local models now refuse over-long prompts, as they should
- The capability matrix checks every answer came from the model asked,
  that models call only offered tools, and that a declared
  supports_tool_choice is true
- Spec A15 and the tuning guide say how enforcement is requested,
  declared and backed up

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

nisse-dev/src/nisse_dev/probe.py

@@ -11,6 +11,10 @@ with it:
 - **It took the whole prompt.** The server allows more than is declared, so
   the declaration is conservative.
 
+Every answer is checked for which model gave it. A proxy that falls back to
+another model on an error returns a normal-looking answer under that model's
+name, and once made this probe report a refusing server as roomy.
+
 Run it with ``scripts/probe_context.py``. It spends one small and one large
 request on the model, so it refuses windows above a ceiling unless told
 otherwise: commercial models' limits are already known to LiteLLM.
@@ -20,6 +24,8 @@ from dataclasses import dataclass
 
 import httpx
 
+from nisse_dev.catalog import ModelFacts
+
 # Of a probe's size, how much must come back for it to count as uncut. A
 # chat template adds a little, and a tokenizer counts a little differently.
 _UNCUT = 0.9
@@ -86,7 +92,14 @@ def _prompt(words: int) -> str:
     return " ".join(f"item{i}" for i in range(words)) + "\n\nReply with just: ok"
 
 
-def _send(http: httpx.Client, model: str, words: int) -> tuple[int | None, str | None]:
+class Fallback(Exception):
+    def __init__(self, answered_by: str) -> None:
+        self.answered_by = answered_by
+
+
+def _send(
+    http: httpx.Client, model: str, words: int, backing: str
+) -> tuple[int | None, str | None]:
     response = http.post(
         "/chat/completions",
         json={
@@ -97,18 +110,33 @@ def _send(http: httpx.Client, model: str, words: int) -> tuple[int | None, str |
     )
     if response.status_code != 200:
         return None, f"{response.status_code}: {response.text}"
-    usage = response.json().get("usage") or {}
+    data = response.json()
+    answered = str(data.get("model") or "")
+    facts = ModelFacts(alias=model, backing=backing, provider="", supports_tools=None)
+    if not facts.answered(answered):
+        raise Fallback(answered)
+    usage = data.get("usage") or {}
     return int(usage.get("prompt_tokens") or 0), None
 
 
-def probe(http: httpx.Client, model: str, declared: int) -> ProbeResult:
+def probe(
+    http: httpx.Client, model: str, declared: int, backing: str = ""
+) -> ProbeResult:
     """Measure how many tokens a word of the probe costs on this model, then
     send a prompt half again as long as the declared window."""
     calibration_words = 200
-    tokens, error = _send(http, model, calibration_words)
-    if error is not None or not tokens:
-        return ProbeResult("failed", f"a short calibration request failed: {error}")
-    per_word = tokens / calibration_words
-    sent = int(declared * 1.5)
-    reported, error = _send(http, model, int(sent / per_word))
+    try:
+        tokens, error = _send(http, model, calibration_words, backing)
+        if error is not None or not tokens:
+            return ProbeResult("failed", f"a short calibration request failed: {error}")
+        per_word = tokens / calibration_words
+        sent = int(declared * 1.5)
+        reported, error = _send(http, model, int(sent / per_word), backing)
+    except Fallback as fallback:
+        return ProbeResult(
+            "fallback",
+            f"{fallback.answered_by} answered instead of {model}: the proxy fell "
+            "back to another model, so nothing here describes this one. Remove "
+            "the fallback, or probe the server directly.",
+        )
     return interpret(declared, sent, reported, error)