dongarandClaude Opus 5
249bcb10fc
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 47s
Measuring Opus found a bug that had been silent all along: nothing checked stop_reason or finish_reason, so a response cut off mid-answer was parsed as a complete one. The symptom was a submit_plan tool call whose required `content` field was simply absent — structurally valid, substantively empty, and indistinguishable from a model that chose to send nothing. The cause was our max_tokens of 4096: Opus writing an architecture plan routinely spends 3,000-4,000 output tokens, and was observed at 4,064 against that ceiling. Roughly one run in three crossed it. Worse than the empty plan is what the same failure does elsewhere: a truncated write_file writes half a file and reports success. - Both routes now raise TruncatedResponse when the model was cut off. It subclasses LLMError, so the planner's retry and the executor's and delegation's gating already handle it without learning a new type: a cut-off turn is a failed turn - The default ceiling is 16384 and configurable via NISSE_MAX_OUTPUT_TOKENS. Output is billed by what is produced, not by the ceiling, so a low limit buys nothing and costs silent truncation Verified: three consecutive Opus planning turns now return plans of 5,700-6,200 characters where one in three previously returned nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>