Detect truncated responses, and raise the output ceiling

Measuring Opus found a bug that had been silent all along: nothing
checked stop_reason or finish_reason, so a response cut off mid-answer
was parsed as a complete one.

The symptom was a submit_plan tool call whose required `content` field
was simply absent — structurally valid, substantively empty, and
indistinguishable from a model that chose to send nothing. The cause
was our max_tokens of 4096: Opus writing an architecture plan
routinely spends 3,000-4,000 output tokens, and was observed at 4,064
against that ceiling. Roughly one run in three crossed it.

Worse than the empty plan is what the same failure does elsewhere: a
truncated write_file writes half a file and reports success.

- Both routes now raise TruncatedResponse when the model was cut off.
  It subclasses LLMError, so the planner's retry and the executor's
  and delegation's gating already handle it without learning a new
  type: a cut-off turn is a failed turn
- The default ceiling is 16384 and configurable via
  NISSE_MAX_OUTPUT_TOKENS. Output is billed by what is produced, not
  by the ceiling, so a low limit buys nothing and costs silent
  truncation

Verified: three consecutive Opus planning turns now return plans of
5,700-6,200 characters where one in three previously returned nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

nisse-dev/src/nisse_dev/llm.py

@@ -41,6 +41,28 @@ class LLMError(RuntimeError):
 RATE_LIMIT_STATUS = 429
 RATE_LIMIT_RETRY_S = 20.0
 
+# Generous by default. Output is billed by what is produced, not by the
+# ceiling, and a ceiling too low does not fail loudly — it truncates a tool
+# call mid-argument and hands back something that parses. Measured: Opus
+# writing an architecture plan routinely spends 3,000-4,000 output tokens and
+# was observed at 4,064 against a 4,096 limit.
+DEFAULT_MAX_OUTPUT_TOKENS = 16384
+
+
+def max_output_tokens(env: Mapping[str, str] = os.environ) -> int:
+    raw = env.get("NISSE_MAX_OUTPUT_TOKENS")
+    return int(raw) if raw else DEFAULT_MAX_OUTPUT_TOKENS
+
+
+class TruncatedResponse(LLMError):
+    """The model was cut off mid-answer.
+
+    Raised rather than returned, because the damage is silent: a truncated
+    tool call arrives with required arguments simply absent, and a truncated
+    write_file would write half a file and report success. A cut-off turn is
+    a failed turn, and every caller already retries or gates on LLMError.
+    """
+
 
 class LLMClient(Protocol):
     def complete(
@@ -131,9 +153,15 @@ class LiteLLMClient:
         self, data: dict[str, Any], requested_model: str
     ) -> list[CanonicalEvent]:
         try:
-            message = data["choices"][0]["message"]
+            choice = data["choices"][0]
+            message = choice["message"]
         except (KeyError, IndexError) as exc:
             raise LLMError(f"malformed completion response: {exc}") from exc
+        if choice.get("finish_reason") == "length":
+            raise TruncatedResponse(
+                f"{requested_model} hit its output limit mid-answer; "
+                "raise NISSE_MAX_OUTPUT_TOKENS"
+            )
         produced_by = data.get("model") or requested_model
         out: list[CanonicalEvent] = []
         if message.get("content"):
@@ -184,7 +212,7 @@ class AnthropicPassthroughClient:
         token: str,
         *,
         timeout_s: float = 300.0,
-        max_tokens: int = 4096,
+        max_tokens: int | None = None,
         transport: httpx.BaseTransport | None = None,
         sleep: Any = time.sleep,
     ) -> None:
@@ -199,7 +227,7 @@ class AnthropicPassthroughClient:
             transport=transport,
         )
         self._renderer = AnthropicMessagesRenderer()
-        self._max_tokens = max_tokens
+        self._max_tokens = max_tokens or max_output_tokens()
         self._sleep = sleep
 
     @classmethod
@@ -234,6 +262,11 @@ class AnthropicPassthroughClient:
     def _parse(
         self, data: dict[str, Any], requested_model: str
     ) -> list[CanonicalEvent]:
+        if data.get("stop_reason") == "max_tokens":
+            raise TruncatedResponse(
+                f"{requested_model} hit the output limit of {self._max_tokens} "
+                f"tokens mid-answer; raise NISSE_MAX_OUTPUT_TOKENS"
+            )
         content = data.get("content")
         if not isinstance(content, list):
             raise LLMError(f"malformed passthrough response: {str(data)[:300]}")