dongarandClaude Opus 5.5
659915d309
Found live on 2026-10-07 (bring-up on nisse-testing-operator/nisse-bringup,
Opus as driver):
- The driver's prose nudge went into the system prompt, so the retry ended
on the model's own reply. Opus 5.5 refuses that ("does not support
assistant message prefill. The conversation must end with a user
message"). The nudge is now a harness fact in the conversation. The
delegation harness's prose path did the same and is fixed likewise.
- A helper's first request ended with its own delegation_started event,
rendered as an assistant message: every delegation would have failed on
Opus. The helper now sees it as the harness handing over its task.
- model_contract checks that every request ends on the user's side, so
scripted tests catch this from now on (each fix is mutation-checked).
- `status` (and `details`) in an item's thread were taken as replies: they
woke the driver and cancelled a bug's countdown. They now answer at once
and change nothing.
- The driver had no way to answer a remark without changing the item, so it
answered in prose. Every run woken by a message (or a review comment) now
offers `reply`, which answers where asked and returns to the very wait
that was interrupted.
Also: scripts/bringup-repo.sh makes the bring-up's test repository
(make_repo/delete_repo, split out of fresh_repo), and made repositories get
the agent's labels (bug, discuss, design).
Tests: test_loop_live_fixes.py (one per finding); the walk sends `status`
into threads (status_changes_nothing) and its scripted driver sometimes
answers remarks with reply.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
66 lines
2.8 KiB
Python
66 lines
2.8 KiB
Python
"""What a real model API insists on, for scripted test models.
|
|
|
|
Both formats the agent sends require a tool call's result to come right
|
|
after it: in Anthropic's, the next message is the user's and starts with a
|
|
tool_result for every tool_use, before any text; in OpenAI's, the assistant
|
|
message with tool_calls is followed directly by a tool message for each. A
|
|
scripted model that doesn't check this lets a broken transcript pass the
|
|
unit tests and be rejected live; this makes it fail here.
|
|
"""
|
|
|
|
from typing import Any
|
|
|
|
from nisse_dev.render import OpenAIChatRenderer
|
|
from nisse_dev.render_anthropic import AnthropicMessagesRenderer
|
|
|
|
|
|
def assert_renderable(events: list[Any]) -> None:
|
|
_anthropic(AnthropicMessagesRenderer().render(list(events), None, "m")["messages"])
|
|
_openai(OpenAIChatRenderer().render(list(events), None, "m")["messages"])
|
|
|
|
|
|
def _anthropic(messages: list[dict[str, Any]]) -> None:
|
|
# A request ends on the user's side: current models refuse assistant
|
|
# prefill ("The conversation must end with a user message"), found live
|
|
# on 2026-10-07 when the driver's prose nudge left its own reply last.
|
|
assert not messages or messages[-1]["role"] == "user", (
|
|
f"anthropic messages.{len(messages) - 1}: the conversation ends with an "
|
|
"assistant message (prefill), which the API refuses"
|
|
)
|
|
for index, message in enumerate(messages):
|
|
if message["role"] != "assistant":
|
|
continue
|
|
used = [b["id"] for b in message["content"] if b.get("type") == "tool_use"]
|
|
if not used:
|
|
continue
|
|
following = messages[index + 1] if index + 1 < len(messages) else None
|
|
assert following is not None and following["role"] == "user", (
|
|
f"anthropic messages.{index}: tool_use with no user message after it"
|
|
)
|
|
blocks = following["content"]
|
|
leading = []
|
|
for block in blocks:
|
|
if block.get("type") != "tool_result":
|
|
break
|
|
leading.append(block["tool_use_id"])
|
|
assert set(used) <= set(leading), (
|
|
f"anthropic messages.{index}: tool_use ids {sorted(set(used) - set(leading))} "
|
|
"lack a tool_result at the start of the next message"
|
|
)
|
|
|
|
|
|
def _openai(messages: list[dict[str, Any]]) -> None:
|
|
for index, message in enumerate(messages):
|
|
calls = [c["id"] for c in message.get("tool_calls") or []]
|
|
if not calls:
|
|
continue
|
|
answered = []
|
|
for after in messages[index + 1 :]:
|
|
if after["role"] != "tool":
|
|
break
|
|
answered.append(after["tool_call_id"])
|
|
assert set(calls) <= set(answered), (
|
|
f"openai messages.{index}: tool_calls {sorted(set(calls) - set(answered))} "
|
|
"are not answered by the tool messages right after them"
|
|
)
|