A model's safety-filter refusal is an error that says so, not an empty reply
Opus's provider filter declined one of nisse-review's ask_expert questions (which repositories a Gitea token can list) every time. It came back as a 200 with no text, and the expert reported "unparseable reply: empty reply". Both LLM clients now raise ModelRefused for finish_reason content_filter (chat route) and stop_reason refusal (passthrough). The expert's handback says what happened and what to do instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
This commit is contained in:
1 parent
098d42ff2c
commit
9d2992c20a
4 files changed
+96
-2
No files matched your search
@@ -32,7 +32,7 @@ from nisse_core.log.canonical import (
|
||||
StoredEvent,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_core.models.llm import LLMClient, LLMError, RoleMap
|
||||
from nisse_core.models.llm import LLMClient, LLMError, ModelRefused, RoleMap
|
||||
from nisse_core.state.store import Transcript
|
||||
|
||||
EXPERT_ROLE = "expert"
|
||||
@@ -168,6 +168,9 @@ class ExpertDesk:
|
||||
model=model,
|
||||
system=system,
|
||||
)
|
||||
except ModelRefused as exc:
|
||||
# Not retried: the same question is declined again.
|
||||
return None, f"{exc}. Ask it differently, or work from the repository"
|
||||
except LLMError as exc:
|
||||
return None, str(exc)
|
||||
text = "\n".join(
|
||||
|
||||
@@ -102,6 +102,24 @@ class TruncatedResponse(LLMError):
|
||||
"""
|
||||
|
||||
|
||||
class ModelRefused(LLMError):
|
||||
"""The provider's safety filter declined to answer.
|
||||
|
||||
It comes back as a successful reply with no text (``finish_reason:
|
||||
content_filter`` on the chat route, ``stop_reason: refusal`` on the
|
||||
passthrough), so unchecked it reads as an empty answer. It can be a false
|
||||
positive: Opus declined, every time, a question about which repositories
|
||||
a Gitea token can list. Asking the same thing again gets the same answer.
|
||||
"""
|
||||
|
||||
|
||||
def _refused(model: str) -> ModelRefused:
|
||||
return ModelRefused(
|
||||
f"{model} declined to answer: its provider's safety filter stopped the "
|
||||
"reply. That says nothing about the subject itself"
|
||||
)
|
||||
|
||||
|
||||
class LLMClient(Protocol):
|
||||
def complete(
|
||||
self,
|
||||
@@ -216,6 +234,8 @@ class LiteLLMClient:
|
||||
f"{requested_model} hit its output limit mid-answer; raise "
|
||||
"llm.max_output_tokens, or the model's max_output_tokens in LiteLLM"
|
||||
)
|
||||
if choice.get("finish_reason") == "content_filter":
|
||||
raise _refused(requested_model)
|
||||
produced_by = data.get("model") or requested_model
|
||||
# What actually answered, which is not always what was asked for: a
|
||||
# proxy fallback answers under another model's name.
|
||||
@@ -338,6 +358,8 @@ class AnthropicPassthroughClient:
|
||||
f"{requested_model} hit its output limit mid-answer; raise "
|
||||
"llm.max_output_tokens, or the model's max_output_tokens in LiteLLM"
|
||||
)
|
||||
if data.get("stop_reason") == "refusal":
|
||||
raise _refused(requested_model)
|
||||
content = data.get("content")
|
||||
if not isinstance(content, list):
|
||||
raise LLMError(f"malformed passthrough response: {str(data)[:300]}")
|
||||
|
||||
Reference in new issue
Block a user