Add ask_expert and eval capture (A14/A16) — step 7c
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m7s
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m7s
Completes step 7. - expert.py: one stateless call for a question needing no repo access. Structured output is contracted at the prompt level, since vendors differ on JSON enforcement, so the reply is parsed and validated locally with one retry — tolerating a fenced code block, because models wrap JSON habitually, but not missing fields - The per-task budget is counted from the canonical log rather than memory, so it survives a restart and cannot hand a circling executor a fresh allowance. Exhaustion raises rather than answering: a caller out of consults should get on with the work or escalate - Consults are recorded as delegation_started/completed with role consultant. A16 asks for a "consult/delegation eval set", and reusing the composite events gets consult evaluation for free rather than inventing a parallel record - evals.py pairs starts with completions into eval records, with verdicts kept outside the canonical log: the log records what happened, a verdict is a later judgement about it, and judgements get revised. Schema v3 adds the outcomes table, so a v1 store now traverses two migrations — a better exercise of the hook than one - ask_expert is offered to the executor as a tool only when a desk is configured, in addition to the observable escalation triggers and never instead of them (A12). Consulting is not mutating, so a completion claim backed only by consults is still challenged Found live: passing an empty message list and putting the question in the system prompt fails outright — the Messages API requires at least one non-system message. The question now travels as a synthesised user message that is never appended to the log, which keeps the consult stateless while remaining a valid request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
dd22eda528
commit
2b7651fb37
9 files changed
+846
-8
No files matched your search
@@ -16,7 +16,7 @@ from nisse_dev.canonical import (
|
||||
from_payload,
|
||||
mint_call_id,
|
||||
)
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.state import CURRENT_SCHEMA_VERSION, StateStore
|
||||
|
||||
ALL_EVENTS = [
|
||||
UserMessage(text="use demo", source="matrix:$abc"),
|
||||
@@ -116,9 +116,12 @@ def test_default_migration_upgrades_a_real_v1_store(tmp_path: Path) -> None:
|
||||
conn.close()
|
||||
|
||||
with StateStore(db) as store:
|
||||
assert store.schema_version == 2
|
||||
assert store.schema_version == CURRENT_SCHEMA_VERSION
|
||||
state = store.get_state()
|
||||
assert state is not None
|
||||
assert state.state == "IDLE" # v1 data intact
|
||||
store.append_event(UserMessage(text="hello", source="m:1"))
|
||||
assert len(store.read_events()) == 1
|
||||
# Two migrations were traversed, so both later tables must exist.
|
||||
store.mark_outcome(1, "good", "arrived via v1 -> v3")
|
||||
assert store.outcomes()[1] == ("good", "arrived via v1 -> v3")
|
||||
Reference in new issue
Block a user