A model's safety-filter refusal is an error that says so, not an empty reply
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m25s
CI / e2e (pull_request) Successful in 4m41s

Opus's provider filter declined one of nisse-review's ask_expert
questions (which repositories a Gitea token can list) every time. It came
back as a 200 with no text, and the expert reported "unparseable reply:
empty reply". Both LLM clients now raise ModelRefused for
finish_reason content_filter (chat route) and stop_reason refusal
(passthrough). The expert's handback says what happened and what to do
instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-10-18 13:39:26 +00:00
1 parent 098d42ff2c
commit 9d2992c20a
4 files changed
+96 -2

No files matched your search

+4 -1
View File
@@ -32,7 +32,7 @@ from nisse_core.log.canonical import (
StoredEvent,
UserMessage,
)
from nisse_core.models.llm import LLMClient, LLMError, RoleMap
from nisse_core.models.llm import LLMClient, LLMError, ModelRefused, RoleMap
from nisse_core.state.store import Transcript
EXPERT_ROLE = "expert"
@@ -168,6 +168,9 @@ class ExpertDesk:
model=model,
system=system,
)
except ModelRefused as exc:
# Not retried: the same question is declined again.
return None, f"{exc}. Ask it differently, or work from the repository"
except LLMError as exc:
return None, str(exc)
text = "\n".join(
+22
View File
@@ -102,6 +102,24 @@ class TruncatedResponse(LLMError):
"""
class ModelRefused(LLMError):
"""The provider's safety filter declined to answer.
It comes back as a successful reply with no text (``finish_reason:
content_filter`` on the chat route, ``stop_reason: refusal`` on the
passthrough), so unchecked it reads as an empty answer. It can be a false
positive: Opus declined, every time, a question about which repositories
a Gitea token can list. Asking the same thing again gets the same answer.
"""
def _refused(model: str) -> ModelRefused:
return ModelRefused(
f"{model} declined to answer: its provider's safety filter stopped the "
"reply. That says nothing about the subject itself"
)
class LLMClient(Protocol):
def complete(
self,
@@ -216,6 +234,8 @@ class LiteLLMClient:
f"{requested_model} hit its output limit mid-answer; raise "
"llm.max_output_tokens, or the model's max_output_tokens in LiteLLM"
)
if choice.get("finish_reason") == "content_filter":
raise _refused(requested_model)
produced_by = data.get("model") or requested_model
# What actually answered, which is not always what was asked for: a
# proxy fallback answers under another model's name.
@@ -338,6 +358,8 @@ class AnthropicPassthroughClient:
f"{requested_model} hit its output limit mid-answer; raise "
"llm.max_output_tokens, or the model's max_output_tokens in LiteLLM"
)
if data.get("stop_reason") == "refusal":
raise _refused(requested_model)
content = data.get("content")
if not isinstance(content, list):
raise LLMError(f"malformed passthrough response: {str(data)[:300]}")