The status page's walkthrough (M6.6b)
A new package, nisse-ui: a read-only page for studying how the agents work. For each work item it shows the steps (what people and the orchestrator said, and each run), each run's turns, and each turn's request read against the log it was made from: - each message beside the events it came from, tagged unchanged, labelled, narrated, shortened (by how much) or a compaction, with what fitting dropped; - what is new since the turn before, and where the request first differs (where prompt reuse had to stop), next to the server's count; - the reply with its reasoning, and the raw JSON. Runs going in circles are flagged by nisse_core.loop.circles (repeated near-identical replies, one correction given again and again, repeated apologies), a pure function M7.8 can reuse in Driver.run. In nisse-core: - render.py: message_indexes(), from the same code render() uses, so the page's matching can't drift from what was sent. - calls.py: the record notes when recording began; the reader gains calls_for(item) and call(id). - state/readonly.py: opens an agent's SQLite file without creating its -wal/-shm files (mode=ro alone does, once the writer has closed). The page reads only through read-only connections, escapes everything, and sends a Content Security Policy that allows no script. It listens on 127.0.0.1; until M6.6d puts it behind Authelia, reach it over an SSH tunnel. Jinja2 is its one new dependency, for autoescaping. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
This commit is contained in:
dongarandClaude Opus 5.5 committed
1 parent
c1f3d9dafb
commit
a8e2023183
31 files changed
+2550
-15
No files matched your search
@@ -0,0 +1,40 @@
|
||||
# nisse-ui
|
||||
|
||||
The status page (architecture X6): a read-only view of what the agents did
|
||||
and how, for studying context, compaction and memory in practice.
|
||||
|
||||
It reads each agent's state store and the call record beside it (L7), and
|
||||
never writes to either. It runs no model and holds no credentials.
|
||||
|
||||
uv run nisse-ui --source dev=/var/lib/nisse-dev/state.db \
|
||||
--source review=/var/lib/nisse-review/state.db \
|
||||
--runs /path/to/kept-runs
|
||||
|
||||
- `--source NAME=STATE_DB`: an agent's state store. Its call record is
|
||||
`<stem>.calls.db` beside it.
|
||||
- `--runs DIR`: runs the benchmark kept (`scripts/benchmark.py run
|
||||
--keep-runs DIR`). Each folder holding a `state.db` is a source.
|
||||
|
||||
It listens on 127.0.0.1:8765, because it shows everything the models saw,
|
||||
including what other people wrote. Until it sits behind Caddy and Authelia
|
||||
(M6.6d), reach it through an SSH tunnel:
|
||||
|
||||
ssh -L 8765:localhost:8765 <this VM>
|
||||
|
||||
then open http://localhost:8765/.
|
||||
|
||||
**What it shows:** for each work item, its steps (what people and the
|
||||
orchestrator said, and each run), each run's turns, and for each turn:
|
||||
|
||||
- the request exactly as sent, message by message, beside the log events
|
||||
each came from and what rendering did to them;
|
||||
- what changed since the turn before, and where prompt reuse had to stop,
|
||||
next to the server's own count;
|
||||
- the reply, with its reasoning;
|
||||
- the raw JSON.
|
||||
|
||||
Runs going in circles are flagged (`nisse_core.loop.circles`).
|
||||
|
||||
**Safety:** everything is escaped, and the page sends a Content Security
|
||||
Policy that runs no script at all. Files are opened through
|
||||
`nisse_core.state.readonly`, which never creates or locks anything.
|
||||
@@ -0,0 +1,16 @@
|
||||
[project]
|
||||
name = "nisse-ui"
|
||||
version = "0.1.0"
|
||||
description = "Status page: a read-only view of what the agents did and how (X6)"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = ["nisse-core", "jinja2>=3.1"]
|
||||
|
||||
[project.scripts]
|
||||
nisse-ui = "nisse_ui.main:main"
|
||||
|
||||
[tool.uv.sources]
|
||||
nisse-core = { workspace = true }
|
||||
|
||||
[build-system]
|
||||
requires = ["uv_build>=0.12,<0.13"]
|
||||
build-backend = "uv_build"
|
||||
@@ -0,0 +1 @@
|
||||
"""nisse-ui: the operator's read-only view of the agents (architecture X6)."""
|
||||
@@ -0,0 +1,230 @@
|
||||
"""The page's routes, as a plain function from a path to a response, so the
|
||||
tests drive it without a socket (X6).
|
||||
|
||||
/ the sources: agents and kept runs
|
||||
/s/<source>/ a source's work items
|
||||
/s/<source>/item/<id> an item's walkthrough ("-": the agent-level log)
|
||||
/s/<source>/call/<n> one turn, read against its log
|
||||
/s/<source>/event/<n> one log event in full
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from dataclasses import dataclass, field
|
||||
from importlib import resources
|
||||
from typing import Any
|
||||
from urllib.parse import quote, unquote, urlsplit
|
||||
|
||||
from jinja2 import Environment, PackageLoader, StrictUndefined
|
||||
|
||||
from nisse_core.models import calls
|
||||
from nisse_ui import walk
|
||||
from nisse_ui.sources import Source, StateReader, UnknownSchema, call_record
|
||||
|
||||
AGENT_LEVEL = "-" # the item id that stands for the agent-level log
|
||||
|
||||
# Everything shown is untrusted text (L11): escaped by the templates, and
|
||||
# the browser runs no script at all, loads nothing from elsewhere, and
|
||||
# sends no referrer.
|
||||
HEADERS = {
|
||||
"Content-Security-Policy": (
|
||||
"default-src 'none'; style-src 'self'; img-src 'self'; base-uri 'none'; "
|
||||
"form-action 'none'; frame-ancestors 'none'"
|
||||
),
|
||||
"X-Content-Type-Options": "nosniff",
|
||||
"Referrer-Policy": "no-referrer",
|
||||
"Cache-Control": "no-store",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Response:
|
||||
status: int
|
||||
body: bytes
|
||||
content_type: str = "text/html; charset=utf-8"
|
||||
headers: dict[str, str] = field(default_factory=lambda: dict(HEADERS))
|
||||
|
||||
|
||||
def _when(ts: Any) -> str:
|
||||
return str(ts)[:19].replace("T", " ") if ts else "—"
|
||||
|
||||
|
||||
def _num(n: Any) -> str:
|
||||
return f"{n:,}" if isinstance(n, int) and not isinstance(n, bool) else "—"
|
||||
|
||||
|
||||
def _pretty(text: Any) -> str:
|
||||
"""JSON text indented, anything else as it is."""
|
||||
if not isinstance(text, str):
|
||||
return json.dumps(text, indent=2, ensure_ascii=False)
|
||||
try:
|
||||
return json.dumps(json.loads(text), indent=2, ensure_ascii=False)
|
||||
except ValueError:
|
||||
return text
|
||||
|
||||
|
||||
def _clip(text: Any, limit: int = 300) -> str:
|
||||
text = str(text or "")
|
||||
return (
|
||||
text
|
||||
if len(text) <= limit
|
||||
else text[:limit] + f" … [{len(text) - limit:,} more]"
|
||||
)
|
||||
|
||||
|
||||
def summary(kind: str, payload: dict[str, Any]) -> str:
|
||||
"""One event as a line of text: what it said or did."""
|
||||
match kind:
|
||||
case "user_message" | "assistant_text" | "fact":
|
||||
return str(payload.get("text", ""))
|
||||
case "tool_call":
|
||||
args = json.dumps(payload.get("arguments", {}), ensure_ascii=False)
|
||||
return f"{payload.get('tool', '?')}({args})"
|
||||
case "tool_result":
|
||||
mark = "ok" if payload.get("success") else "FAILED"
|
||||
return f"{mark}: {payload.get('result', '')}"
|
||||
case "plan_submitted":
|
||||
return str(payload.get("plan", ""))
|
||||
case "gate_opened":
|
||||
return f"{payload.get('gate', '')}: {payload.get('reason', '')}"
|
||||
case "delegation_started":
|
||||
return f"({payload.get('model', '')}) {payload.get('brief', '')}"
|
||||
case "delegation_completed":
|
||||
return f"[{payload.get('exit_status', '')}] {payload.get('handback', '')}"
|
||||
case "compaction":
|
||||
first, last = (payload.get("replaces") or ["?", "?"])[:2]
|
||||
return f"replaces {first}–{last}: {payload.get('summary', '')}"
|
||||
case "action_verdict":
|
||||
return (
|
||||
f"{payload.get('tool', '')}: {payload.get('effect', '')} → "
|
||||
f"{payload.get('decision', '')} ({payload.get('reason', '')})"
|
||||
)
|
||||
case _:
|
||||
return json.dumps(payload, ensure_ascii=False)
|
||||
|
||||
|
||||
def _reply_line(reply: walk.Reply) -> str:
|
||||
calls_made = "; ".join(f"{tool}({args})" for tool, args in reply.calls)
|
||||
text = " ".join(reply.text.split())
|
||||
return " · ".join(p for p in (text, calls_made) if p) or "(no reply)"
|
||||
|
||||
|
||||
class App:
|
||||
def __init__(self, sources: list[Source]) -> None:
|
||||
self.sources = list(sources)
|
||||
self.by_name = {s.name: s for s in sources}
|
||||
env = Environment(
|
||||
loader=PackageLoader("nisse_ui", "templates"),
|
||||
autoescape=True, # always: every page shows untrusted text
|
||||
undefined=StrictUndefined,
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
)
|
||||
env.filters.update(
|
||||
u=lambda s: quote(str(s), safe=""),
|
||||
when=_when,
|
||||
num=_num,
|
||||
pretty=_pretty,
|
||||
clip=_clip,
|
||||
reply_line=_reply_line,
|
||||
)
|
||||
env.globals.update(summary=summary, AGENT_LEVEL=AGENT_LEVEL)
|
||||
self.env = env
|
||||
|
||||
def page(self, template: str, status: int = 200, **values: Any) -> Response:
|
||||
html = self.env.get_template(template).render(**values)
|
||||
return Response(status, html.encode())
|
||||
|
||||
def problem(self, status: int, title: str, detail: str) -> Response:
|
||||
return self.page("problem.html", status, title=title, detail=detail)
|
||||
|
||||
def handle(self, raw_path: str) -> Response:
|
||||
path = urlsplit(raw_path).path
|
||||
parts = [unquote(p) for p in path.split("/")[1:]]
|
||||
if parts in ([], [""]):
|
||||
return self.home()
|
||||
if parts == ["static", "style.css"]:
|
||||
css = resources.files("nisse_ui").joinpath("static/style.css").read_bytes()
|
||||
return Response(200, css, "text/css; charset=utf-8")
|
||||
if len(parts) >= 2 and parts[0] == "s":
|
||||
source = self.by_name.get(parts[1])
|
||||
if source is None:
|
||||
return self.problem(404, "No such source", parts[1])
|
||||
rest = parts[2:]
|
||||
try:
|
||||
if rest in ([], [""]):
|
||||
return self.items(source)
|
||||
if len(rest) == 2 and rest[0] == "item":
|
||||
item = None if rest[1] == AGENT_LEVEL else rest[1]
|
||||
return self.item(source, item)
|
||||
if (
|
||||
len(rest) == 2
|
||||
and rest[0] in ("call", "event")
|
||||
and rest[1].isdigit()
|
||||
):
|
||||
if rest[0] == "call":
|
||||
return self.call(source, int(rest[1]))
|
||||
return self.event(source, int(rest[1]))
|
||||
except UnknownSchema as exc:
|
||||
return self.problem(
|
||||
409, "A state store this page doesn't know", str(exc)
|
||||
)
|
||||
except calls.UnknownLayout as exc:
|
||||
return self.problem(
|
||||
409, "A call record this page doesn't know", str(exc)
|
||||
)
|
||||
except sqlite3.Error as exc:
|
||||
return self.problem(503, "Couldn't read the store", str(exc))
|
||||
return self.problem(404, "Not found", path)
|
||||
|
||||
# --- pages ---------------------------------------------------------------------
|
||||
|
||||
def home(self) -> Response:
|
||||
rows = []
|
||||
for source in self.sources:
|
||||
row: dict[str, Any] = {"source": source, "count": None, "note": ""}
|
||||
try:
|
||||
with StateReader(source.state_db) as state:
|
||||
row["count"] = state.count_items()
|
||||
row["schema"] = state.schema_version
|
||||
except (UnknownSchema, sqlite3.Error) as exc:
|
||||
row["note"] = str(exc)
|
||||
row["record"] = source.calls_db.is_file()
|
||||
rows.append(row)
|
||||
return self.page("sources.html", rows=rows)
|
||||
|
||||
def items(self, source: Source) -> Response:
|
||||
with StateReader(source.state_db) as state:
|
||||
return self.page(
|
||||
"items.html",
|
||||
source=source,
|
||||
items=state.items(),
|
||||
active=state.active_item(),
|
||||
)
|
||||
|
||||
def item(self, source: Source, item_id: str | None) -> Response:
|
||||
with StateReader(source.state_db) as state:
|
||||
if item_id is not None and state.item(item_id) is None:
|
||||
return self.problem(404, "No such work item", item_id)
|
||||
try:
|
||||
with call_record(source) as reader:
|
||||
view = walk.walkthrough(state, reader, item_id)
|
||||
except calls.UnknownLayout as exc:
|
||||
view = walk.walkthrough(state, None, item_id, record_note=str(exc))
|
||||
return self.page("item.html", source=source, view=view)
|
||||
|
||||
def call(self, source: Source, call_id: int) -> Response:
|
||||
with StateReader(source.state_db) as state, call_record(source) as reader:
|
||||
if reader is None:
|
||||
return self.problem(404, "No call record", str(source.calls_db))
|
||||
view = walk.turn(state, reader, call_id)
|
||||
if view is None:
|
||||
return self.problem(404, "No such call", str(call_id))
|
||||
return self.page("call.html", source=source, view=view)
|
||||
|
||||
def event(self, source: Source, event_id: int) -> Response:
|
||||
with StateReader(source.state_db) as state:
|
||||
found = state.event(event_id)
|
||||
if found is None:
|
||||
return self.problem(404, "No such event", str(event_id))
|
||||
return self.page("event.html", source=source, event=found)
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Serve the status page (X6), read-only, on this machine only.
|
||||
|
||||
nisse-ui --source dev=/var/lib/nisse-dev/state.db --runs bench-kept/
|
||||
|
||||
Each source is an agent's state store; its call record is found beside it.
|
||||
``--runs`` names folders of runs the benchmark kept (``--keep-runs``). It
|
||||
listens on 127.0.0.1: until it sits behind Authelia (M6.6d), reach it
|
||||
through an SSH tunnel, since it shows everything the models saw.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_ui.app import App
|
||||
from nisse_ui.sources import discover
|
||||
|
||||
|
||||
def _source(text: str) -> tuple[str, Path]:
|
||||
name, sep, path = text.partition("=")
|
||||
if not sep or not name or not path:
|
||||
raise argparse.ArgumentTypeError("expected NAME=STATE_DB")
|
||||
return name, Path(path)
|
||||
|
||||
|
||||
def handler_for(app: App) -> type[BaseHTTPRequestHandler]:
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
server_version = "nisse-ui"
|
||||
sys_version = ""
|
||||
|
||||
def do_GET(self) -> None:
|
||||
response = app.handle(self.path)
|
||||
self.send_response(response.status)
|
||||
self.send_header("Content-Type", response.content_type)
|
||||
self.send_header("Content-Length", str(len(response.body)))
|
||||
for key, value in response.headers.items():
|
||||
self.send_header(key, value)
|
||||
self.end_headers()
|
||||
self.wfile.write(response.body)
|
||||
|
||||
def log_message(self, format: str, *args: object) -> None:
|
||||
sys.stderr.write(f"nisse-ui {self.address_string()} {format % args}\n")
|
||||
|
||||
return Handler
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument(
|
||||
"--source", action="append", type=_source, default=[], metavar="NAME=STATE_DB"
|
||||
)
|
||||
parser.add_argument("--runs", action="append", type=Path, default=[], metavar="DIR")
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, default=8765)
|
||||
args = parser.parse_args(argv)
|
||||
app = App(discover(args.source, args.runs))
|
||||
server = ThreadingHTTPServer((args.host, args.port), handler_for(app))
|
||||
print(f"nisse-ui on http://{args.host}:{args.port}/", file=sys.stderr)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,229 @@
|
||||
"""Where the page reads from (X6): each agent's state store and the call
|
||||
record beside it, and the runs the benchmark kept. Everything is opened
|
||||
read-only (SQLite ``mode=ro``), one short connection per page, so the agent
|
||||
stays the only writer of its state (L8) and nothing here can lock it."""
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Self
|
||||
|
||||
from nisse_core.models import calls
|
||||
from nisse_core.state import readonly
|
||||
|
||||
# The state store schema versions this page knows: the shared store's, and
|
||||
# each agent's that adds tables of its own. The page reads only the shared
|
||||
# tables, but says so for any version it doesn't know rather than guessing.
|
||||
KNOWN_SCHEMAS = {4: "nisse-core (nisse-review)", 5: "nisse-dev"}
|
||||
|
||||
|
||||
class UnknownSchema(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Source:
|
||||
name: str # what the page calls it, safe in a URL
|
||||
state_db: Path
|
||||
kind: str # "agent" | "run" (a kept benchmark run)
|
||||
|
||||
@property
|
||||
def calls_db(self) -> Path:
|
||||
return calls.path_for(self.state_db)
|
||||
|
||||
|
||||
def safe_name(text: str) -> str:
|
||||
return re.sub(r"[^A-Za-z0-9._-]+", "-", text).strip("-") or "source"
|
||||
|
||||
|
||||
def discover(agents: list[tuple[str, Path]], runs: list[Path]) -> list[Source]:
|
||||
"""The agents named on the command line, then each run kept under the
|
||||
runs folders (a folder holding a ``state.db``), newest first."""
|
||||
found = [Source(safe_name(name), path, "agent") for name, path in agents]
|
||||
for folder in runs:
|
||||
kept = sorted(
|
||||
(p for p in folder.glob("*/state.db") if p.is_file()),
|
||||
key=lambda p: p.stat().st_mtime,
|
||||
reverse=True,
|
||||
)
|
||||
found += [
|
||||
Source(safe_name(f"{folder.name}.{p.parent.name}"), p, "run") for p in kept
|
||||
]
|
||||
names: set[str] = set()
|
||||
unique = []
|
||||
for source in found:
|
||||
name, n = source.name, 2
|
||||
while name in names:
|
||||
name, n = f"{source.name}-{n}", n + 1
|
||||
names.add(name)
|
||||
unique.append(Source(name, source.state_db, source.kind))
|
||||
return unique
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Item:
|
||||
id: str
|
||||
title: str
|
||||
status: str
|
||||
outcome: str | None
|
||||
position: str
|
||||
scope: str | None
|
||||
opened_at: str
|
||||
updated_at: str
|
||||
closed_at: str | None
|
||||
data: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Event:
|
||||
id: int
|
||||
ts: str
|
||||
type: str
|
||||
payload: dict[str, Any]
|
||||
work_item: str | None
|
||||
actor: str
|
||||
|
||||
|
||||
def _loads(text: str | None) -> Any:
|
||||
"""Item data and payloads, read tolerantly: whatever doesn't parse is kept
|
||||
as text, so a field from a newer agent never breaks the page."""
|
||||
if text is None:
|
||||
return None
|
||||
try:
|
||||
return json.loads(text)
|
||||
except (TypeError, ValueError):
|
||||
return {"unreadable": text}
|
||||
|
||||
|
||||
def connect(path: Path) -> sqlite3.Connection:
|
||||
return readonly.connect(path)
|
||||
|
||||
|
||||
class StateReader:
|
||||
"""A state store, read-only. Use as a context manager: it closes at the
|
||||
end of the page that opened it."""
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self.path = path
|
||||
self._conn = connect(path)
|
||||
try:
|
||||
row = self._conn.execute(
|
||||
"SELECT value FROM meta WHERE key = 'schema_version'"
|
||||
).fetchone()
|
||||
except sqlite3.Error as exc:
|
||||
self._conn.close()
|
||||
raise UnknownSchema(f"not a state store: {exc}") from exc
|
||||
self.schema_version = int(row["value"]) if row else 0
|
||||
if self.schema_version not in KNOWN_SCHEMAS:
|
||||
self._conn.close()
|
||||
known = ", ".join(str(v) for v in sorted(KNOWN_SCHEMAS))
|
||||
raise UnknownSchema(
|
||||
f"state store schema {self.schema_version}, not known to this "
|
||||
f"page (it knows {known})"
|
||||
)
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> None:
|
||||
self.close()
|
||||
|
||||
def close(self) -> None:
|
||||
self._conn.close()
|
||||
|
||||
def active_item(self) -> str | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT active_item FROM agent WHERE id = 1"
|
||||
).fetchone()
|
||||
return row["active_item"] if row else None
|
||||
|
||||
def items(self) -> list[Item]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM work_items ORDER BY opened_at DESC, id"
|
||||
).fetchall()
|
||||
return [self._item(r) for r in rows]
|
||||
|
||||
def item(self, item_id: str) -> Item | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM work_items WHERE id = ?", (item_id,)
|
||||
).fetchone()
|
||||
return self._item(row) if row else None
|
||||
|
||||
def _item(self, row: sqlite3.Row) -> Item:
|
||||
data = _loads(row["data"])
|
||||
return Item(
|
||||
id=row["id"],
|
||||
title=row["title"],
|
||||
status=row["status"],
|
||||
outcome=row["outcome"],
|
||||
position=row["position"],
|
||||
scope=row["scope"],
|
||||
opened_at=row["opened_at"],
|
||||
updated_at=row["updated_at"],
|
||||
closed_at=row["closed_at"],
|
||||
data=data if isinstance(data, dict) else {"unreadable": row["data"]},
|
||||
)
|
||||
|
||||
def events(self, work_item: str | None) -> list[Event]:
|
||||
"""One item's log, or the agent-level log (None)."""
|
||||
if work_item is None:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item IS NULL ORDER BY id"
|
||||
)
|
||||
else:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item = ? ORDER BY id",
|
||||
(work_item,),
|
||||
)
|
||||
return [self._event(r) for r in rows]
|
||||
|
||||
def event(self, event_id: int) -> Event | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE id = ?", (event_id,)
|
||||
).fetchone()
|
||||
return self._event(row) if row else None
|
||||
|
||||
def events_by_id(self, ids: list[int]) -> dict[int, Event]:
|
||||
found: dict[int, Event] = {}
|
||||
wanted = sorted({i for i in ids if i > 0})
|
||||
for start in range(0, len(wanted), 500):
|
||||
chunk = wanted[start : start + 500]
|
||||
marks = ",".join("?" * len(chunk))
|
||||
for row in self._conn.execute(
|
||||
f"SELECT * FROM canonical_events WHERE id IN ({marks})", chunk
|
||||
):
|
||||
found[row["id"]] = self._event(row)
|
||||
return found
|
||||
|
||||
def count_items(self) -> int:
|
||||
return self._conn.execute("SELECT COUNT(*) FROM work_items").fetchone()[0]
|
||||
|
||||
def _event(self, row: sqlite3.Row) -> Event:
|
||||
payload = _loads(row["payload"])
|
||||
return Event(
|
||||
id=row["id"],
|
||||
ts=row["ts"],
|
||||
type=row["type"],
|
||||
payload=payload if isinstance(payload, dict) else {"value": payload},
|
||||
work_item=row["work_item"],
|
||||
actor=row["actor"],
|
||||
)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def call_record(source: Source) -> Iterator[calls.CallReader | None]:
|
||||
"""The source's call record, or None when there is none (an agent
|
||||
started before M6.6a, or recording turned off). An unknown layout raises
|
||||
``calls.UnknownLayout`` for the page to report."""
|
||||
if not source.calls_db.is_file():
|
||||
yield None
|
||||
return
|
||||
reader = calls.CallReader(source.calls_db)
|
||||
try:
|
||||
yield reader
|
||||
finally:
|
||||
reader.close()
|
||||
@@ -0,0 +1,77 @@
|
||||
/* nisse-ui: plain, readable, no external assets (X6). */
|
||||
:root {
|
||||
--bg: #fbfbfa; --fg: #1d1d1b; --muted: #6b6b66; --line: #deded8;
|
||||
--panel: #f2f2ee; --accent: #2f5d8a; --bad: #a3322a; --warn: #8a5a00;
|
||||
--new: #e3f0e0; --change: #fbe6c8; --flag: #f6d9d6;
|
||||
--mono: ui-monospace, "SF Mono", Menlo, Consolas, monospace;
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #161615; --fg: #e6e6e1; --muted: #9a9a93; --line: #34342f;
|
||||
--panel: #20201e; --accent: #8fb8e0; --bad: #ec8a80; --warn: #e2b462;
|
||||
--new: #233523; --change: #3d2f17; --flag: #432522;
|
||||
}
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; background: var(--bg); color: var(--fg);
|
||||
font: 15px/1.5 system-ui, -apple-system, "Segoe UI", sans-serif; }
|
||||
main { max-width: 1100px; margin: 0 auto; padding: 0 16px 64px; }
|
||||
a { color: var(--accent); }
|
||||
h1 { font-size: 1.5rem; margin: 1.2rem 0 .4rem; }
|
||||
h2 { font-size: 1.15rem; margin: 1.8rem 0 .6rem; border-bottom: 1px solid var(--line); }
|
||||
h3 { font-size: 1rem; margin: 1.2rem 0 .4rem; }
|
||||
.top { border-bottom: 1px solid var(--line); background: var(--panel); }
|
||||
.crumbs { max-width: 1100px; margin: 0 auto; padding: 8px 16px; }
|
||||
.muted, .meta { color: var(--muted); }
|
||||
.meta { font-size: .9rem; }
|
||||
.bad { color: var(--bad); }
|
||||
.path { font-family: var(--mono); font-size: .85rem; word-break: break-all; }
|
||||
pre, code { font-family: var(--mono); font-size: .85rem; }
|
||||
pre { background: var(--panel); border: 1px solid var(--line); border-radius: 4px;
|
||||
padding: 8px 10px; overflow-x: auto; white-space: pre-wrap; word-break: break-word; margin: .4rem 0; }
|
||||
table { border-collapse: collapse; width: 100%; margin: .6rem 0; font-size: .92rem; }
|
||||
th, td { text-align: left; vertical-align: top; padding: 4px 8px; border-bottom: 1px solid var(--line); }
|
||||
th { color: var(--muted); font-weight: 600; }
|
||||
tr.active td { background: var(--new); }
|
||||
table.facts th { width: 14%; white-space: nowrap; }
|
||||
table.log td.text { font-family: var(--mono); font-size: .8rem; word-break: break-word; }
|
||||
details { margin: .4rem 0; }
|
||||
summary { cursor: pointer; }
|
||||
.tag { display: inline-block; font-size: .78rem; padding: 0 6px; border-radius: 3px;
|
||||
background: var(--panel); border: 1px solid var(--line); margin-right: 2px; }
|
||||
.tag.warn { color: var(--warn); }
|
||||
.flag { display: inline-block; font-size: .78rem; padding: 0 6px; border-radius: 3px;
|
||||
background: var(--flag); color: var(--bad); margin-left: 4px; }
|
||||
.badge { display: inline-block; font-size: .78rem; padding: 0 6px; border-radius: 3px; }
|
||||
.badge.new { background: var(--new); }
|
||||
.badge.change { background: var(--change); }
|
||||
.note { background: var(--panel); border-left: 3px solid var(--accent); padding: 6px 10px; }
|
||||
.circle { background: var(--flag); color: var(--bad); padding: 4px 10px; border-radius: 4px; }
|
||||
.timeline > * { margin: .5rem 0; }
|
||||
.step { border-left: 3px solid var(--line); padding: 2px 10px; }
|
||||
.step .text { white-space: pre-wrap; word-break: break-word; }
|
||||
.step-user_message { border-left-color: var(--accent); }
|
||||
.step-gate_opened, .step-fact { border-left-color: var(--warn); }
|
||||
.run { border: 1px solid var(--line); border-radius: 6px; padding: 4px 10px; background: var(--bg); }
|
||||
.run .run { margin: .4rem 0 .4rem 12px; }
|
||||
.run > summary { padding: 4px 0; }
|
||||
.turn { padding: 4px 0 4px 8px; border-top: 1px dashed var(--line); }
|
||||
.turn.flagged { background: var(--flag); }
|
||||
.turn-no { font-weight: 600; margin-right: 6px; }
|
||||
.turn .reply { font-family: var(--mono); font-size: .82rem; color: var(--muted); word-break: break-word; }
|
||||
.told { font-size: .82rem; color: var(--warn); }
|
||||
.pager { display: flex; justify-content: space-between; margin-top: 1rem; }
|
||||
.msg { border: 1px solid var(--line); border-radius: 6px; padding: 6px 10px; margin: .6rem 0; }
|
||||
.msg.first-change { border-color: var(--warn); border-width: 2px; }
|
||||
.msg-head { font-size: .88rem; }
|
||||
.msg-head .role { font-weight: 700; text-transform: uppercase; letter-spacing: .03em; margin-right: 4px; }
|
||||
.role-user { border-left: 4px solid var(--accent); }
|
||||
.role-assistant { border-left: 4px solid #6a8f4e; }
|
||||
.role-tool { border-left: 4px solid var(--muted); }
|
||||
.role-system { border-left: 4px solid var(--warn); }
|
||||
ul.from { margin: .3rem 0; padding-left: 1.2rem; font-size: .85rem; }
|
||||
ul.from .text { font-family: var(--mono); font-size: .8rem; color: var(--muted); }
|
||||
.toolcall { margin: .3rem 0; }
|
||||
@media (max-width: 640px) {
|
||||
table.facts th, table.facts td { display: block; width: auto; border: 0; }
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>{% block title %}Agents{% endblock %} · nisse-ui</title>
|
||||
<link rel="stylesheet" href="/static/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header class="top">
|
||||
<nav class="crumbs"><a href="/">nisse-ui</a>{% block crumbs %}{% endblock %}</nav>
|
||||
</header>
|
||||
<main>
|
||||
{% block main %}{% endblock %}
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,93 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "macros.html" import src_link, item_href, event_link, flags %}
|
||||
{% set c = view.call %}
|
||||
{% block title %}Turn {{ view.number }}{% endblock %}
|
||||
{% block crumbs %} › {{ src_link(source) }} › <a href="{{ item_href(source, c.work_item) }}">{{ c.work_item or "agent" }}</a> › {% if view.run %}{{ view.run.kind }} {{ view.run.role }} › {% endif %}turn {{ view.number }}{% endblock %}
|
||||
{% block main %}
|
||||
<nav class="pager">
|
||||
{% if view.previous %}<a href="/s/{{ source.name|u }}/call/{{ view.previous }}">← turn {{ view.number - 1 }}</a>{% else %}<span class="muted">first turn</span>{% endif %}
|
||||
{% if view.following %}<a href="/s/{{ source.name|u }}/call/{{ view.following }}">turn {{ view.number + 1 }} →</a>{% else %}<span class="muted">last turn</span>{% endif %}
|
||||
</nav>
|
||||
<h1>Turn {{ view.number }}{% if c.purpose %}: {{ c.purpose }}{% endif %} {{ flags(view.flags) }}</h1>
|
||||
|
||||
<table class="facts">
|
||||
<tr><th>Role</th><td>{{ c.role or "—" }}</td><th>Time</th><td>{{ c.at|when }}, {{ "%.2f"|format(c.seconds) }}s</td></tr>
|
||||
<tr><th>Model asked</th><td>{{ c.model }}</td><th>Answered by</th><td>{{ c.answered_by or "—" }}</td></tr>
|
||||
<tr><th>Tokens</th><td>{{ c.tokens_in|num }} in · {{ c.tokens_out|num }} out</td><th>Prompt reuse (server)</th><td>{{ view.reused or "not reported" }}</td></tr>
|
||||
<tr><th>Estimate</th><td>{{ c.estimate|num }} tokens (factor {{ c.factor }})</td><th>Route</th><td>{{ c.route or "—" }}, HTTP {{ c.status or "—" }}</td></tr>
|
||||
<tr><th>Outcome</th><td{% if c.outcome != "answered" %} class="bad"{% endif %}>{{ c.outcome }}</td><th>Fitting</th><td>{% if c.trim %}{{ c.trim|pretty }}{% else %}nothing cut{% endif %}</td></tr>
|
||||
</table>
|
||||
{% if c.error %}<pre class="bad">{{ c.error }}</pre>{% endif %}
|
||||
|
||||
<h2>The reply</h2>
|
||||
{% if view.reply.reasoning %}
|
||||
<details open class="reasoning"><summary>Reasoning ({{ view.reply.reasoning|length|num }} characters)</summary><pre class="text">{{ view.reply.reasoning }}</pre></details>
|
||||
{% endif %}
|
||||
{% if view.reply.text %}<pre class="text">{{ view.reply.text }}</pre>{% endif %}
|
||||
{% for tool, args in view.reply.calls %}
|
||||
<div class="toolcall"><span class="tag">tool call</span> <b>{{ tool }}</b><pre>{{ args|pretty }}</pre></div>
|
||||
{% endfor %}
|
||||
{% if not view.reply.text and not view.reply.calls and not view.reply.reasoning %}<p class="muted">No reply recorded.</p>{% endif %}
|
||||
{% if view.reply.finish %}<p class="meta">Finish reason: {{ view.reply.finish }}</p>{% endif %}
|
||||
|
||||
<h2>The request</h2>
|
||||
{% if view.settings %}
|
||||
<table class="facts">{% for k, v in view.settings.items() %}<tr><th>{{ k }}</th><td><code>{{ v|pretty }}</code></td></tr>{% endfor %}</table>
|
||||
{% endif %}
|
||||
{% if view.tools %}
|
||||
<details><summary>{{ view.tools|length }} tools offered{% if view.tools_changed %} <span class="flag">changed since the turn before</span>{% endif %}</summary>
|
||||
<p>{{ view.tools|join(", ") }}</p></details>
|
||||
{% endif %}
|
||||
{% if view.compared %}
|
||||
<p class="note">Messages marked <span class="badge new">new</span> weren't in the turn before. <span class="badge change">first change</span> marks where this request first differs from the last one: prompt reuse can't go past it. The server reused {{ view.reused or "an unreported number of" }} tokens.</p>
|
||||
{% endif %}
|
||||
{% if view.match_note %}<p class="note">{{ view.match_note }}</p>{% endif %}
|
||||
|
||||
{% for m in view.messages %}
|
||||
<section class="msg role-{{ m.role }}{% if m.first_change %} first-change{% endif %}" id="m{{ m.index }}">
|
||||
<div class="msg-head">
|
||||
<span class="role">{{ m.role }}</span>{% if m.index >= 0 %} <span class="muted">#{{ m.index }}</span>{% endif %}
|
||||
{% if m.new %}<span class="badge new">new</span>{% endif %}
|
||||
{% if m.first_change %}<span class="badge change">first change</span>{% endif %}
|
||||
{% if m.tool_call_id %}<span class="muted">result of {{ m.tool_call_id }}</span>{% endif %}
|
||||
<span class="muted">{{ m.text|length|num }} characters</span>
|
||||
</div>
|
||||
{% if m.role == "system" %}
|
||||
<details><summary>System prompt</summary><pre class="text">{{ m.text }}</pre></details>
|
||||
{% elif m.text %}<pre class="text">{{ m.text }}</pre>{% endif %}
|
||||
{% for id, tool, args in m.tool_calls %}
|
||||
<div class="toolcall"><span class="tag">tool call</span> <b>{{ tool }}</b> <span class="muted">{{ id }}</span><pre>{{ args|pretty }}</pre></div>
|
||||
{% endfor %}
|
||||
{% if m.events %}
|
||||
<ul class="from">
|
||||
{% for s in m.events %}<li>from {{ event_link(source, s.id) }} {{ s.type|replace("_", " ") }} by {{ s.actor }}: {% for t in s.tags %}<span class="tag{% if t != 'unchanged' %} warn{% endif %}">{{ t }}</span>{% endfor %}</li>{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
</section>
|
||||
{% endfor %}
|
||||
|
||||
{% if view.unmatched %}
|
||||
<h3>Events sent{% if view.match_note %}, not matched to messages{% else %} but replaced by a compaction{% endif %}</h3>
|
||||
<ul class="from">
|
||||
{% for s in view.unmatched %}<li>{{ event_link(source, s.id) }} {{ s.type|replace("_", " ") }} by {{ s.actor }}: {% for t in s.tags %}<span class="tag{% if t != 'unchanged' %} warn{% endif %}">{{ t }}</span>{% endfor %}
|
||||
<div class="text">{{ summary(s.type, s.payload if s.payload is mapping else {})|clip(300) }}</div></li>{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
|
||||
{% if view.compacted %}
|
||||
<h3>Replaced by compaction</h3>
|
||||
<p>{{ view.compacted|length }} events of the log stood behind a compaction in this request:
|
||||
{% for e, comp in view.compacted %}{{ event_link(source, e) }}{% if comp %} → {{ event_link(source, comp) }}{% endif %}{% if not loop.last %}, {% endif %}{% endfor %}</p>
|
||||
{% endif %}
|
||||
|
||||
{% if view.dropped %}
|
||||
<h3>Dropped by fitting</h3>
|
||||
<ul class="from">
|
||||
{% for id, e in view.dropped %}<li>{{ event_link(source, id) }}{% if e %} {{ e.type|replace("_", " ") }} by {{ e.actor }}: {{ summary(e.type, e.payload)|clip(200) }}{% else %} <span class="muted">(not in this store)</span>{% endif %}</li>{% endfor %}
|
||||
</ul>
|
||||
{% endif %}
|
||||
|
||||
<h2>Raw</h2>
|
||||
<details><summary>Request JSON, exactly as sent ({{ view.raw_request|length|num }} characters)</summary><pre>{{ view.raw_request or "(not recorded)" }}</pre></details>
|
||||
<details><summary>Response JSON, as received ({{ view.raw_response|length|num }} characters)</summary><pre>{{ view.raw_response or "(not recorded)" }}</pre></details>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,10 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "macros.html" import src_link, item_href %}
|
||||
{% block title %}Event {{ event.id }}{% endblock %}
|
||||
{% block crumbs %} › {{ src_link(source) }} › <a href="{{ item_href(source, event.work_item) }}">{{ event.work_item or "agent" }}</a> › event {{ event.id }}{% endblock %}
|
||||
{% block main %}
|
||||
<h1>Event #{{ event.id }}: {{ event.type|replace("_", " ") }}</h1>
|
||||
<p class="meta">{{ event.ts|when }} · by {{ event.actor }} · work item {{ event.work_item or "(agent-level)" }}</p>
|
||||
<pre class="text">{{ summary(event.type, event.payload) }}</pre>
|
||||
<details><summary>Payload</summary><pre>{{ event.payload|pretty }}</pre></details>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,37 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "macros.html" import src_link, run_node, turn_row, step, event_link %}
|
||||
{% set title = view.item.title if view.item else "Agent-level log" %}
|
||||
{% block title %}{{ title }}{% endblock %}
|
||||
{% block crumbs %} › {{ src_link(source) }} › {{ view.item.id if view.item else "agent" }}{% endblock %}
|
||||
{% block main %}
|
||||
<h1>{{ title }}</h1>
|
||||
{% if view.item %}
|
||||
<p class="meta">{{ view.item.id }} · {{ view.item.status }}{% if view.item.outcome %} ({{ view.item.outcome }}){% endif %}
|
||||
· position {{ view.item.position }} · opened {{ view.item.opened_at|when }}{% if view.item.closed_at %} · closed {{ view.item.closed_at|when }}{% endif %}</p>
|
||||
<details><summary>Item data</summary><pre>{{ view.item.data|pretty }}</pre></details>
|
||||
{% endif %}
|
||||
|
||||
{% if view.record_note %}<p class="note">{{ view.record_note }}</p>
|
||||
{% elif view.before_record %}<p class="note">Recording began {{ view.started_at|when }}: steps before then have no requests.</p>{% endif %}
|
||||
|
||||
<h2>Steps</h2>
|
||||
<div class="timeline">
|
||||
{% for kind, obj in view.timeline %}
|
||||
{% if kind == "event" %}{{ step(source, obj) }}
|
||||
{% elif kind == "run" %}{{ run_node(source, obj) }}
|
||||
{% else %}{{ turn_row(source, obj) }}{% endif %}
|
||||
{% else %}
|
||||
<p class="muted">Nothing recorded yet.</p>
|
||||
{% endfor %}
|
||||
</div>
|
||||
|
||||
<h2>The log</h2>
|
||||
<details{% if view.record_note %} open{% endif %}>
|
||||
<summary>All {{ view.log|length }} events, as stored</summary>
|
||||
<table class="log">
|
||||
{% for e in view.log %}
|
||||
<tr><td>{{ event_link(source, e.id) }}</td><td>{{ e.ts|when }}</td><td>{{ e.actor }}</td><td>{{ e.type }}</td><td class="text">{{ summary(e.type, e.payload)|clip(200) }}</td></tr>
|
||||
{% endfor %}
|
||||
</table>
|
||||
</details>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,26 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "macros.html" import item_href %}
|
||||
{% block title %}{{ source.name }}{% endblock %}
|
||||
{% block crumbs %} › {{ source.name }}{% endblock %}
|
||||
{% block main %}
|
||||
<h1>{{ source.name }}</h1>
|
||||
<p class="muted path">{{ source.state_db }}</p>
|
||||
<p><a href="{{ item_href(source, none) }}">Agent-level log</a>: what happened outside any work item.</p>
|
||||
<table>
|
||||
<thead><tr><th>Work item</th><th>Title</th><th>Status</th><th>Position</th><th>Opened</th><th>Updated</th></tr></thead>
|
||||
<tbody>
|
||||
{% for item in items %}
|
||||
<tr{% if item.id == active %} class="active"{% endif %}>
|
||||
<td><a href="{{ item_href(source, item.id) }}">{{ item.id }}</a>{% if item.id == active %} <span class="tag">active</span>{% endif %}</td>
|
||||
<td>{{ item.title }}</td>
|
||||
<td>{{ item.status }}{% if item.outcome %} ({{ item.outcome }}){% endif %}</td>
|
||||
<td>{{ item.position }}</td>
|
||||
<td>{{ item.opened_at|when }}</td>
|
||||
<td>{{ item.updated_at|when }}</td>
|
||||
</tr>
|
||||
{% else %}
|
||||
<tr><td colspan="6" class="muted">No work items.</td></tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,48 @@
|
||||
{% macro src_link(source) %}<a href="/s/{{ source.name|u }}/">{{ source.name }}</a>{% endmacro %}
|
||||
|
||||
{% macro item_href(source, item_id) %}/s/{{ source.name|u }}/item/{{ (item_id if item_id is not none else AGENT_LEVEL)|u }}{% endmacro %}
|
||||
|
||||
{% macro event_link(source, id) %}{% if id > 0 %}<a href="/s/{{ source.name|u }}/event/{{ id }}">#{{ id }}</a>{% else %}<span class="muted">(no id)</span>{% endif %}{% endmacro %}
|
||||
|
||||
{% macro flags(names) %}{% for f in names %}<span class="flag">{{ f }}</span>{% endfor %}{% endmacro %}
|
||||
|
||||
{% macro turn_row(source, t) %}
|
||||
<div class="turn{% if t.flags %} flagged{% endif %}">
|
||||
<a class="turn-no" href="/s/{{ source.name|u }}/call/{{ t.call.id }}">turn {{ t.number }}</a>
|
||||
<span class="meta">
|
||||
{% if t.call.purpose %}<span class="tag">{{ t.call.purpose }}</span>{% endif %}
|
||||
{{ t.call.role or "" }} · {{ t.call.model }}{% if t.call.answered_by and t.call.answered_by != t.call.model %} → {{ t.call.answered_by }}{% endif %}
|
||||
· {{ "%.1f"|format(t.call.seconds) }}s · {{ t.call.tokens_in|num }} in / {{ t.call.tokens_out|num }} out
|
||||
{% if t.reused %} · reused {{ t.reused }}{% endif %}
|
||||
{% if t.call.outcome != "answered" %} · <span class="bad">{{ t.call.outcome }}</span>{% endif %}
|
||||
</span>
|
||||
{{ flags(t.flags) }}
|
||||
{% for told in t.told %}<div class="told">orchestrator: {{ told|clip(160) }}</div>{% endfor %}
|
||||
<div class="reply">{{ t.reply|reply_line|clip(240) }}</div>
|
||||
</div>
|
||||
{% endmacro %}
|
||||
|
||||
{% macro run_node(source, node) %}
|
||||
<details class="run" open>
|
||||
<summary>
|
||||
<span class="tag">{{ node.run.kind }}</span> {{ node.run.role }}
|
||||
· {{ node.run.outcome or "running" }}
|
||||
· {{ node.run.started_at|when }} → {{ node.run.ended_at|when }}
|
||||
· {{ node.turns|length }} turns
|
||||
{% if node.run.note %}<span class="muted"> · {{ node.run.note|clip(160) }}</span>{% endif %}
|
||||
</summary>
|
||||
{% for c in node.circles %}
|
||||
<p class="circle">Going in circles: {{ c.kind }} at turns {% for i in c.turns %}{{ i + 1 }}{% if not loop.last %}, {% endif %}{% endfor %}: {{ c.detail }}</p>
|
||||
{% endfor %}
|
||||
{% for kind, obj in node.entries %}
|
||||
{% if kind == "turn" %}{{ turn_row(source, obj) }}{% elif kind == "event" %}{{ step(source, obj) }}{% else %}{{ run_node(source, obj) }}{% endif %}
|
||||
{% endfor %}
|
||||
</details>
|
||||
{% endmacro %}
|
||||
|
||||
{% macro step(source, e) %}
|
||||
<div class="step step-{{ e.type }}">
|
||||
<span class="meta">{{ event_link(source, e.id) }} {{ e.ts|when }} · {{ e.actor }} · <span class="tag">{{ e.type|replace("_", " ") }}</span></span>
|
||||
<div class="text">{{ summary(e.type, e.payload)|clip(600) }}</div>
|
||||
</div>
|
||||
{% endmacro %}
|
||||
@@ -0,0 +1,6 @@
|
||||
{% extends "base.html" %}
|
||||
{% block title %}{{ title }}{% endblock %}
|
||||
{% block main %}
|
||||
<h1>{{ title }}</h1>
|
||||
<p class="note">{{ detail }}</p>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,21 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "macros.html" import src_link %}
|
||||
{% block title %}Sources{% endblock %}
|
||||
{% block main %}
|
||||
<h1>Agents and recorded runs</h1>
|
||||
{% if not rows %}<p class="muted">No sources. Start with <code>--source NAME=STATE_DB</code> or <code>--runs DIR</code>.</p>{% endif %}
|
||||
<table>
|
||||
<thead><tr><th>Source</th><th>Kind</th><th>Work items</th><th>Call record</th><th>State store</th></tr></thead>
|
||||
<tbody>
|
||||
{% for r in rows %}
|
||||
<tr>
|
||||
<td>{{ src_link(r.source) }}</td>
|
||||
<td>{{ r.source.kind }}</td>
|
||||
<td>{% if r.count is not none %}{{ r.count }}{% else %}<span class="bad">{{ r.note }}</span>{% endif %}</td>
|
||||
<td>{{ "yes" if r.record else "none" }}</td>
|
||||
<td class="path">{{ r.source.state_db }}</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,543 @@
|
||||
"""What the pages show, built from the two stores (X6): an item's
|
||||
walkthrough (its steps, runs and turns) and one turn read against the log it
|
||||
was made from. Pure reading: nothing here writes."""
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from nisse_core.log.canonical import StoredEvent, from_payload
|
||||
from nisse_core.loop.circles import Circle, Turn, find_circles
|
||||
from nisse_core.models import calls
|
||||
from nisse_core.models.render import OpenAIChatRenderer
|
||||
from nisse_ui.sources import Event, Item, StateReader
|
||||
|
||||
# The log events shown as steps between runs: what people and the
|
||||
# orchestrator said and decided. A model's own turns are shown as turns.
|
||||
STEP_TYPES = {
|
||||
"user_message",
|
||||
"fact",
|
||||
"gate_opened",
|
||||
"plan_submitted",
|
||||
"delegation_started",
|
||||
"delegation_completed",
|
||||
"compaction",
|
||||
}
|
||||
_TURN_TYPES = {"assistant_text", "tool_call", "tool_result", "action_verdict"}
|
||||
|
||||
|
||||
# --- a model's reply ------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Reply:
|
||||
text: str = ""
|
||||
reasoning: str = ""
|
||||
calls: tuple[tuple[str, str], ...] = () # (tool, arguments as sent)
|
||||
finish: str | None = None
|
||||
|
||||
|
||||
def _text(content: Any) -> str:
|
||||
if content is None:
|
||||
return ""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for part in content:
|
||||
if isinstance(part, dict) and isinstance(part.get("text"), str):
|
||||
parts.append(part["text"])
|
||||
else:
|
||||
parts.append(json.dumps(part, ensure_ascii=False))
|
||||
return "\n".join(parts)
|
||||
return json.dumps(content, ensure_ascii=False)
|
||||
|
||||
|
||||
def _arguments(value: Any) -> str:
|
||||
return value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
|
||||
|
||||
|
||||
def reply_of(response: Any) -> Reply:
|
||||
"""The reply in a raw response, from either route: the OpenAI-shaped chat
|
||||
route or Anthropic's messages route. Anything else is no reply."""
|
||||
if not isinstance(response, dict):
|
||||
return Reply()
|
||||
choices = response.get("choices")
|
||||
if isinstance(choices, list) and choices and isinstance(choices[0], dict):
|
||||
message = choices[0].get("message") or {}
|
||||
reasoning = message.get("reasoning_content") or message.get("reasoning") or ""
|
||||
tool_calls = []
|
||||
for call in message.get("tool_calls") or []:
|
||||
function = (call.get("function") or {}) if isinstance(call, dict) else {}
|
||||
tool_calls.append(
|
||||
(str(function.get("name", "?")), _arguments(function.get("arguments")))
|
||||
)
|
||||
return Reply(
|
||||
text=_text(message.get("content")),
|
||||
reasoning=_text(reasoning),
|
||||
calls=tuple(tool_calls),
|
||||
finish=choices[0].get("finish_reason"),
|
||||
)
|
||||
blocks = response.get("content")
|
||||
if isinstance(blocks, list):
|
||||
texts, thinking, tool_calls = [], [], []
|
||||
for block in blocks:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
match block.get("type"):
|
||||
case "text":
|
||||
texts.append(str(block.get("text", "")))
|
||||
case "thinking":
|
||||
thinking.append(str(block.get("thinking", "")))
|
||||
case "tool_use":
|
||||
tool_calls.append(
|
||||
(str(block.get("name", "?")), _arguments(block.get("input")))
|
||||
)
|
||||
return Reply(
|
||||
text="\n".join(texts),
|
||||
reasoning="\n".join(thinking),
|
||||
calls=tuple(tool_calls),
|
||||
finish=response.get("stop_reason"),
|
||||
)
|
||||
return Reply()
|
||||
|
||||
|
||||
# --- an item's walkthrough --------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class TurnRow:
|
||||
"""One call, as a line of the walkthrough."""
|
||||
|
||||
call: dict[str, Any]
|
||||
number: int # within its run, from 1
|
||||
reply: Reply
|
||||
told: tuple[str, ...] # corrections given since the turn before
|
||||
flags: list[str] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def reused(self) -> str | None:
|
||||
return reuse(self.call)
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunNode:
|
||||
run: dict[str, Any]
|
||||
entries: list[tuple[str, Any]] = field(default_factory=list) # turns, runs
|
||||
turns: list[TurnRow] = field(default_factory=list)
|
||||
circles: list[Circle] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Walkthrough:
|
||||
item: Item | None # None: the agent-level log
|
||||
timeline: list[
|
||||
tuple[str, Any]
|
||||
] # ("event", Event) | ("run", RunNode) | ("turn", TurnRow)
|
||||
log: list[Event]
|
||||
record_note: str | None # why there are no requests, when there aren't
|
||||
started_at: str | None
|
||||
before_record: bool # some of this work happened before recording began
|
||||
|
||||
|
||||
def reuse(call: dict[str, Any]) -> str | None:
|
||||
"""Prompt reuse as the server counted it: cached tokens of the prompt."""
|
||||
cached, fresh = call.get("cache_n"), call.get("prompt_n")
|
||||
if isinstance(cached, int) and isinstance(fresh, int):
|
||||
return f"{cached:,} of {cached + fresh:,}"
|
||||
return None
|
||||
|
||||
|
||||
def _corrections(
|
||||
reader: calls.CallReader, call: dict[str, Any], before: set[int]
|
||||
) -> tuple[str, ...]:
|
||||
"""The orchestrator's statements first sent in this call: a correction
|
||||
is appended to the log each time it's given, so it is new each time."""
|
||||
told = []
|
||||
for event_id, actor, kind, digest in call["sent"]:
|
||||
if kind == "fact" and actor == "orchestrator" and event_id not in before:
|
||||
try:
|
||||
told.append(str(reader.blob(digest).get("text", "")))
|
||||
except (KeyError, AttributeError):
|
||||
continue
|
||||
return tuple(told)
|
||||
|
||||
|
||||
def _response(reader: calls.CallReader, call: dict[str, Any]) -> Any:
|
||||
try:
|
||||
return reader.response(call)
|
||||
except KeyError:
|
||||
return None
|
||||
|
||||
|
||||
def walkthrough(
|
||||
state: StateReader,
|
||||
reader: calls.CallReader | None,
|
||||
item_id: str | None,
|
||||
*,
|
||||
record_note: str | None = None,
|
||||
) -> Walkthrough:
|
||||
item = state.item(item_id) if item_id is not None else None
|
||||
log = state.events(item_id)
|
||||
steps = [e for e in log if e.type in STEP_TYPES or e.type not in _TURN_TYPES]
|
||||
timeline: list[tuple[str, str, Any]] = []
|
||||
started_at = None
|
||||
if reader is None:
|
||||
record_note = record_note or "No call record: requests weren't recorded."
|
||||
else:
|
||||
started_at = reader.started_at
|
||||
nodes = {
|
||||
r["id"]: RunNode(r) for r in reader.runs() if r["work_item"] == item_id
|
||||
}
|
||||
seen: dict[str | None, set[int]] = {}
|
||||
numbers: dict[str | None, int] = {}
|
||||
loose: list[TurnRow] = []
|
||||
for call in reader.calls_for(item_id):
|
||||
run = call["run"] if call["run"] in nodes else None
|
||||
before = seen.setdefault(run, set())
|
||||
numbers[run] = numbers.get(run, 0) + 1
|
||||
row = TurnRow(
|
||||
call=call,
|
||||
number=numbers[run],
|
||||
reply=reply_of(_response(reader, call)),
|
||||
told=_corrections(reader, call, before),
|
||||
)
|
||||
before.update(entry[0] for entry in call["sent"])
|
||||
if run is None:
|
||||
loose.append(row)
|
||||
else:
|
||||
nodes[run].turns.append(row)
|
||||
# A step taken while a run was going on (a correction, a fact) is
|
||||
# shown inside the innermost run it happened during.
|
||||
inside: dict[str, list[tuple[str, str, Any]]] = {}
|
||||
for e in steps:
|
||||
during = [n for n in nodes.values() if _during(n.run, e.ts)]
|
||||
if during:
|
||||
innermost = max(during, key=lambda n: n.run["started_at"])
|
||||
inside.setdefault(innermost.run["id"], []).append(("event", e.ts, e))
|
||||
else:
|
||||
timeline.append(("event", e.ts, e))
|
||||
steps = []
|
||||
for node in nodes.values():
|
||||
node.circles = find_circles(
|
||||
[Turn(t.reply.text, t.reply.calls, t.told) for t in node.turns]
|
||||
)
|
||||
for circle in node.circles:
|
||||
for i in circle.turns:
|
||||
node.turns[i].flags.append(circle.kind)
|
||||
children = [n for n in nodes.values() if n.run["parent"] == node.run["id"]]
|
||||
entries = [("turn", t.call["at"], t) for t in node.turns]
|
||||
entries += [("run", c.run["started_at"], c) for c in children]
|
||||
entries += inside.get(node.run["id"], [])
|
||||
node.entries = [(kind, obj) for kind, _, obj in sorted(entries, key=_when)]
|
||||
for node in nodes.values():
|
||||
if node.run["parent"] not in nodes:
|
||||
timeline.append(("run", node.run["started_at"], node))
|
||||
timeline += [("turn", t.call["at"], t) for t in loose]
|
||||
timeline += [("event", e.ts, e) for e in steps]
|
||||
ordered = [(kind, obj) for kind, _, obj in sorted(timeline, key=_when)]
|
||||
before_record = reader is None or (
|
||||
started_at is not None and bool(log) and log[0].ts < started_at
|
||||
)
|
||||
return Walkthrough(item, ordered, log, record_note, started_at, before_record)
|
||||
|
||||
|
||||
def _during(run: dict[str, Any], ts: str) -> bool:
|
||||
ended = run.get("ended_at")
|
||||
return run["started_at"] <= ts and (ended is None or ts <= ended)
|
||||
|
||||
|
||||
def _when(entry: tuple[str, str, Any]) -> str:
|
||||
return entry[1] or ""
|
||||
|
||||
|
||||
# --- one turn, read against its log -----------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class SentEvent:
|
||||
id: int
|
||||
actor: str
|
||||
type: str
|
||||
payload: Any
|
||||
tags: list[str]
|
||||
log: Event | None
|
||||
|
||||
|
||||
@dataclass
|
||||
class Message:
|
||||
index: int
|
||||
role: str
|
||||
text: str
|
||||
tool_calls: list[tuple[str, str, str]] # (id, tool, arguments)
|
||||
tool_call_id: str | None
|
||||
events: list[SentEvent] = field(default_factory=list)
|
||||
new: bool = False # not in the turn before
|
||||
first_change: bool = False # where this request first differs from the last
|
||||
|
||||
|
||||
@dataclass
|
||||
class TurnView:
|
||||
call: dict[str, Any]
|
||||
number: int
|
||||
run: dict[str, Any] | None
|
||||
previous: int | None
|
||||
following: int | None
|
||||
request: dict[str, Any] | None
|
||||
response: Any
|
||||
reply: Reply
|
||||
messages: list[Message]
|
||||
unmatched: list[SentEvent] # sent events not matched to a message
|
||||
match_note: str | None
|
||||
dropped: list[tuple[int, Event | None]]
|
||||
compacted: list[tuple[int, int | None]]
|
||||
tools: list[str]
|
||||
tools_changed: bool
|
||||
settings: dict[str, Any] # the request's other fields: tool_choice, limits
|
||||
compared: bool # whether there was a turn before to compare with
|
||||
flags: list[str]
|
||||
raw_request: str
|
||||
raw_response: str
|
||||
|
||||
@property
|
||||
def reused(self) -> str | None:
|
||||
return reuse(self.call)
|
||||
|
||||
|
||||
def _fields(call: dict[str, Any] | None) -> dict[str, Any]:
|
||||
if not call or not call.get("request"):
|
||||
return {}
|
||||
try:
|
||||
return dict(json.loads(call["request"])["fields"])
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return {}
|
||||
|
||||
|
||||
def _tags(
|
||||
entry: list[Any], payload: Any, log: Event | None, call: dict[str, Any]
|
||||
) -> list[str]:
|
||||
event_id, _, kind, _ = entry
|
||||
tags: list[str] = []
|
||||
cut = call["shortened"].get(str(event_id))
|
||||
if event_id == 0:
|
||||
tags.append("made for this call, not in the log")
|
||||
elif kind == "compaction" and isinstance(payload, dict):
|
||||
first, last = (payload.get("replaces") or [None, None])[:2]
|
||||
tags.append(f"compaction of events {first}–{last}")
|
||||
elif log is None:
|
||||
tags.append("not in this state store")
|
||||
elif log.payload == payload and log.type == kind:
|
||||
tags.append("unchanged")
|
||||
else:
|
||||
# A role's view says another's work as a labelled statement (L6):
|
||||
# a tool step narrated, a fact or someone's message labelled.
|
||||
source = payload.get("source", "") if isinstance(payload, dict) else ""
|
||||
viewed = kind == "user_message" and str(source).startswith("view:")
|
||||
was = f"{log.type.replace('_', ' ')} by {log.actor}"
|
||||
if viewed and log.type in ("tool_call", "tool_result"):
|
||||
tags.append(f"narrated from {was}")
|
||||
elif viewed:
|
||||
tags.append(
|
||||
"labelled for this role"
|
||||
+ ("" if log.type == "user_message" else f" (a {was})")
|
||||
)
|
||||
elif log.type != kind:
|
||||
tags.append(f"rewritten from {was}")
|
||||
elif cut is None:
|
||||
tags.append("changed for this call")
|
||||
if cut is not None:
|
||||
tags.append(f"shortened by {int(cut):,} characters")
|
||||
return tags
|
||||
|
||||
|
||||
def turn(state: StateReader, reader: calls.CallReader, call_id: int) -> TurnView | None:
|
||||
call = reader.call(call_id)
|
||||
if call is None:
|
||||
return None
|
||||
run = next((r for r in reader.runs() if r["id"] == call["run"]), None)
|
||||
siblings = (
|
||||
reader.calls(run=call["run"])
|
||||
if call["run"]
|
||||
else [c for c in reader.calls_for(call["work_item"]) if not c["run"]]
|
||||
)
|
||||
at = next(i for i, c in enumerate(siblings) if c["id"] == call_id)
|
||||
before = siblings[at - 1] if at > 0 else None
|
||||
after = siblings[at + 1] if at + 1 < len(siblings) else None
|
||||
|
||||
try:
|
||||
request = reader.request(call)
|
||||
except KeyError:
|
||||
request = None
|
||||
response = _response(reader, call)
|
||||
reply = reply_of(response)
|
||||
|
||||
logged = state.events_by_id([entry[0] for entry in call["sent"]])
|
||||
dropped = state.events_by_id(call["dropped"])
|
||||
sent: list[SentEvent] = []
|
||||
for entry in call["sent"]:
|
||||
try:
|
||||
payload = reader.blob(entry[3])
|
||||
except KeyError:
|
||||
payload = None
|
||||
log = logged.get(entry[0])
|
||||
sent.append(
|
||||
SentEvent(
|
||||
entry[0],
|
||||
entry[1],
|
||||
entry[2],
|
||||
payload,
|
||||
_tags(entry, payload, log, call),
|
||||
log,
|
||||
)
|
||||
)
|
||||
|
||||
raw_messages = (request or {}).get("messages")
|
||||
messages: list[Message] = []
|
||||
if isinstance(raw_messages, list):
|
||||
for i, m in enumerate(raw_messages):
|
||||
m = m if isinstance(m, dict) else {"role": "?", "content": m}
|
||||
calls_made = []
|
||||
for c in m.get("tool_calls") or []:
|
||||
function = (c.get("function") or {}) if isinstance(c, dict) else {}
|
||||
calls_made.append(
|
||||
(
|
||||
str(c.get("id", "")) if isinstance(c, dict) else "",
|
||||
str(function.get("name", "?")),
|
||||
_arguments(function.get("arguments")),
|
||||
)
|
||||
)
|
||||
if isinstance(m.get("content"), list): # Anthropic content blocks
|
||||
for block in m["content"]:
|
||||
if isinstance(block, dict) and block.get("type") == "tool_use":
|
||||
calls_made.append(
|
||||
(
|
||||
str(block.get("id", "")),
|
||||
str(block.get("name", "?")),
|
||||
_arguments(block.get("input")),
|
||||
)
|
||||
)
|
||||
messages.append(
|
||||
Message(
|
||||
index=i,
|
||||
role=str(m.get("role", "?")),
|
||||
text=_text(m.get("content")),
|
||||
tool_calls=calls_made,
|
||||
tool_call_id=m.get("tool_call_id"),
|
||||
)
|
||||
)
|
||||
|
||||
unmatched, note = _match(call, request, sent, messages)
|
||||
|
||||
mine, theirs = _fields(call), _fields(before)
|
||||
compared = bool(theirs)
|
||||
if compared:
|
||||
old = theirs.get("messages") or []
|
||||
new = mine.get("messages") or []
|
||||
old_set = set(old)
|
||||
for message, digest in zip(messages, new, strict=False):
|
||||
message.new = digest not in old_set
|
||||
change = next(
|
||||
(i for i, (a, b) in enumerate(zip(old, new, strict=False)) if a != b),
|
||||
min(len(old), len(new)),
|
||||
)
|
||||
if change < len(messages):
|
||||
messages[change].first_change = True
|
||||
|
||||
tools_raw = (request or {}).get("tools") or []
|
||||
tools = []
|
||||
for t in tools_raw if isinstance(tools_raw, list) else []:
|
||||
if isinstance(t, dict):
|
||||
tools.append(
|
||||
str((t.get("function") or {}).get("name") or t.get("name", "?"))
|
||||
)
|
||||
settings = {
|
||||
k: v
|
||||
for k, v in (request or {}).items()
|
||||
if k not in ("messages", "tools", "system")
|
||||
}
|
||||
if isinstance(request, dict) and "system" in request:
|
||||
messages.insert(
|
||||
0,
|
||||
Message(
|
||||
index=-1,
|
||||
role="system",
|
||||
text=_text(request["system"]),
|
||||
tool_calls=[],
|
||||
tool_call_id=None,
|
||||
),
|
||||
)
|
||||
|
||||
flags = []
|
||||
if run is not None:
|
||||
node_turns = [
|
||||
Turn(r.text, r.calls, ())
|
||||
for r in (reply_of(_response(reader, c)) for c in siblings)
|
||||
]
|
||||
flags = sorted({c.kind for c in find_circles(node_turns) if at in c.turns})
|
||||
|
||||
return TurnView(
|
||||
call=call,
|
||||
number=at + 1,
|
||||
run=run,
|
||||
previous=before["id"] if before else None,
|
||||
following=after["id"] if after else None,
|
||||
request=request,
|
||||
response=response,
|
||||
reply=reply,
|
||||
messages=messages,
|
||||
unmatched=unmatched,
|
||||
match_note=note,
|
||||
dropped=[(i, dropped.get(i)) for i in call["dropped"]],
|
||||
compacted=[(int(k), v) for k, v in call["compacted"].items()],
|
||||
tools=tools,
|
||||
tools_changed=compared and mine.get("tools") != theirs.get("tools"),
|
||||
settings=settings,
|
||||
compared=compared,
|
||||
flags=flags,
|
||||
raw_request=json.dumps(request, indent=2, ensure_ascii=False)
|
||||
if request is not None
|
||||
else "",
|
||||
raw_response=json.dumps(response, indent=2, ensure_ascii=False)
|
||||
if response is not None
|
||||
else "",
|
||||
)
|
||||
|
||||
|
||||
def _match(
|
||||
call: dict[str, Any],
|
||||
request: dict[str, Any] | None,
|
||||
sent: list[SentEvent],
|
||||
messages: list[Message],
|
||||
) -> tuple[list[SentEvent], str | None]:
|
||||
"""Each sent event beside the message it went into, found with the same
|
||||
renderer that built the request; when that can't be done exactly, the
|
||||
events are listed apart, with why."""
|
||||
if not sent:
|
||||
return [], None
|
||||
if request is None:
|
||||
return sent, "The request wasn't recorded (the call failed before it)."
|
||||
if call.get("route") != "chat":
|
||||
return sent, "Messages-route requests are shown beside their events, unmatched."
|
||||
try:
|
||||
events = [
|
||||
StoredEvent(id=s.id, ts="", event=from_payload(s.type, s.payload))
|
||||
for s in sent
|
||||
]
|
||||
except (ValueError, TypeError):
|
||||
return sent, "Some events didn't read as this version's event types."
|
||||
first = messages[0] if messages else None
|
||||
system = first.text if first is not None and first.role == "system" else None
|
||||
renderer = OpenAIChatRenderer()
|
||||
expected = renderer.render(events, None, "m", system)["messages"]
|
||||
if len(expected) != len(messages):
|
||||
return sent, (
|
||||
f"Rendering the sent events gives {len(expected)} messages, the request "
|
||||
f"has {len(messages)}; shown apart."
|
||||
)
|
||||
unmatched = []
|
||||
for s, index in zip(sent, renderer.message_indexes(events, system), strict=True):
|
||||
if index is None:
|
||||
unmatched.append(s)
|
||||
else:
|
||||
messages[index].events.append(s)
|
||||
return unmatched, None
|
||||
@@ -0,0 +1,102 @@
|
||||
"""The page and the agents are released together (X6): these fail in the PR
|
||||
that would break the page, not in the operator's browser."""
|
||||
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from ui_fixtures import recorded, store_at
|
||||
|
||||
from nisse_core.models import calls
|
||||
from nisse_core.state.store import StateStore as CoreStore
|
||||
from nisse_dev.state import StateStore as DevStore
|
||||
from nisse_review.state import StateStore as ReviewStore
|
||||
from nisse_ui import main as ui_main
|
||||
from nisse_ui.app import App
|
||||
from nisse_ui.sources import KNOWN_SCHEMAS, discover
|
||||
|
||||
|
||||
@pytest.mark.parametrize("store", [CoreStore, DevStore, ReviewStore])
|
||||
def test_every_agents_schema_is_one_the_page_knows(store: type[CoreStore]) -> None:
|
||||
# An agent that moves its schema on must teach the page too.
|
||||
assert store.SCHEMA_VERSION in KNOWN_SCHEMAS
|
||||
|
||||
|
||||
@pytest.mark.parametrize("store_class", [DevStore, ReviewStore])
|
||||
def test_each_agents_own_store_opens(
|
||||
tmp_path: Path, store_class: type[CoreStore]
|
||||
) -> None:
|
||||
store = store_at(tmp_path, store_class)
|
||||
store.open_work_item("i1", title="Agent item", position="NEW", cause="t")
|
||||
store.close()
|
||||
app = App(discover([("a", tmp_path / "state.db")], []))
|
||||
assert app.handle("/s/a/").status == 200
|
||||
assert "Agent item" in app.handle("/s/a/item/i1").body.decode()
|
||||
|
||||
|
||||
def test_runs_the_benchmark_kept_are_sources(tmp_path: Path) -> None:
|
||||
# As bench --keep-runs leaves them: <dir>/<task>-<id>/state.db + record.
|
||||
for name in ("fix-bug-1a2b3c4d", "trace-bug-5e6f7a8b"):
|
||||
_, store = recorded(tmp_path / "kept" / name)
|
||||
store.close()
|
||||
(tmp_path / "kept" / "not-a-run").mkdir()
|
||||
sources = discover([], [tmp_path / "kept"])
|
||||
assert sorted(s.name for s in sources) == [
|
||||
"kept.fix-bug-1a2b3c4d",
|
||||
"kept.trace-bug-5e6f7a8b",
|
||||
]
|
||||
assert all(s.kind == "run" and s.calls_db.is_file() for s in sources)
|
||||
assert sources[0].calls_db == calls.path_for(sources[0].state_db)
|
||||
|
||||
|
||||
def test_source_names_are_safe_and_unique(tmp_path: Path) -> None:
|
||||
sources = discover(
|
||||
[("dev agent/1", tmp_path / "a"), ("dev agent/1", tmp_path / "b")], []
|
||||
)
|
||||
assert [s.name for s in sources] == ["dev-agent-1", "dev-agent-1-2"]
|
||||
|
||||
|
||||
def test_the_server_listens_on_this_machine_and_sends_the_headers(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
source, store = recorded(tmp_path / "a")
|
||||
store.close()
|
||||
server = ui_main.ThreadingHTTPServer(
|
||||
("127.0.0.1", 0), ui_main.handler_for(App([source]))
|
||||
)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
port = server.server_address[1]
|
||||
response = httpx.get(f"http://127.0.0.1:{port}/s/agent/item/w1")
|
||||
assert response.status_code == 200
|
||||
assert "Fix the parser" in response.text
|
||||
assert "default-src 'none'" in response.headers["content-security-policy"]
|
||||
assert response.headers["x-content-type-options"] == "nosniff"
|
||||
assert response.headers["server"].startswith("nisse-ui")
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
|
||||
|
||||
def test_the_command_line_defaults_to_this_machine_only(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
assert ui_main._source("dev=/var/lib/nisse-dev/state.db") == (
|
||||
"dev",
|
||||
Path("/var/lib/nisse-dev/state.db"),
|
||||
)
|
||||
with pytest.raises(Exception, match="NAME=STATE_DB"):
|
||||
ui_main._source("no-equals")
|
||||
|
||||
class Bound(Exception):
|
||||
pass
|
||||
|
||||
def server(address: tuple[str, int], handler: object) -> None:
|
||||
raise Bound(address)
|
||||
|
||||
monkeypatch.setattr(ui_main, "ThreadingHTTPServer", server)
|
||||
with pytest.raises(Bound) as bound:
|
||||
ui_main.main([])
|
||||
assert bound.value.args[0] == ("127.0.0.1", 8765)
|
||||
@@ -0,0 +1,316 @@
|
||||
"""The walkthrough (X6, M6.6b): an item's steps, its runs' turns, and each
|
||||
turn's request read against the log it was made from."""
|
||||
|
||||
import hashlib
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from ui_fixtures import TOKEN, circling, recorded
|
||||
|
||||
from nisse_core.log.canonical import UserMessage
|
||||
from nisse_core.models import calls
|
||||
from nisse_ui import walk
|
||||
from nisse_ui.app import App
|
||||
from nisse_ui.sources import Source, StateReader, call_record
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def agent(tmp_path: Path) -> Source:
|
||||
source, store = recorded(tmp_path / "agent")
|
||||
store.close()
|
||||
return source
|
||||
|
||||
|
||||
def page(app: App, path: str, status: int = 200) -> str:
|
||||
response = app.handle(path)
|
||||
assert response.status == status, response.body.decode()[:500]
|
||||
return response.body.decode()
|
||||
|
||||
|
||||
def view(source: Source, call_id: int) -> walk.TurnView:
|
||||
with StateReader(source.state_db) as state, call_record(source) as reader:
|
||||
assert reader is not None
|
||||
found = walk.turn(state, reader, call_id)
|
||||
assert found is not None
|
||||
return found
|
||||
|
||||
|
||||
def steps(source: Source, item: str | None) -> walk.Walkthrough:
|
||||
with StateReader(source.state_db) as state, call_record(source) as reader:
|
||||
return walk.walkthrough(state, reader, item)
|
||||
|
||||
|
||||
# --- the way in -------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_sources_and_an_agents_items_are_listed(agent: Source) -> None:
|
||||
app = App([agent])
|
||||
home = page(app, "/")
|
||||
assert "/s/agent/" in home and ">1<" in home # one work item
|
||||
items = page(app, "/s/agent/")
|
||||
assert "Fix the parser" in items and "/s/agent/item/w1" in items
|
||||
assert "/s/agent/item/-" in items # the agent-level log
|
||||
|
||||
|
||||
# --- the walkthrough --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_items_steps_hold_its_runs_turns_and_what_the_orchestrator_said(
|
||||
agent: Source,
|
||||
) -> None:
|
||||
found = steps(agent, "w1")
|
||||
kinds = [kind for kind, _ in found.timeline]
|
||||
assert kinds == ["event", "event", "run"] # the restatement, "go", the run
|
||||
[run] = [obj for kind, obj in found.timeline if kind == "run"]
|
||||
assert run.run["outcome"] == "wait"
|
||||
assert [t.number for t in run.turns] == [1, 2, 3]
|
||||
# The correction for the prose reply came during the run: it's shown
|
||||
# inside it, between the turns it fell between.
|
||||
inner = [kind for kind, _ in run.entries]
|
||||
assert inner == ["turn", "turn", "event", "turn"]
|
||||
assert any("prose" in told for told in run.turns[2].told)
|
||||
assert run.turns[0].reply.calls == (("read_file", '{"path": "parser.py"}'),)
|
||||
assert run.turns[1].reused == "280 of 340"
|
||||
html = page(App([agent]), "/s/agent/item/w1")
|
||||
assert "turn 3" in html and "reused 330 of 350" in html
|
||||
|
||||
|
||||
def test_agent_level_calls_have_their_own_log(agent: Source) -> None:
|
||||
found = steps(agent, None)
|
||||
[run] = [obj for kind, obj in found.timeline if kind == "run"]
|
||||
assert (run.run["kind"], run.run["role"]) == ("classification", "classifier")
|
||||
assert "harmless" in page(App([agent]), "/s/agent/item/-")
|
||||
|
||||
|
||||
def test_work_from_before_recording_has_its_steps_but_no_requests(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
source, store = recorded(tmp_path / "a")
|
||||
# The agent restarts: a fresh, empty record; the log persists.
|
||||
calls.CallRecorder(source.calls_db).close()
|
||||
store.close()
|
||||
found = steps(source, "w1")
|
||||
assert found.before_record
|
||||
# The restatement, "go" and the correction: steps, with no runs to hold them.
|
||||
assert [kind for kind, _ in found.timeline] == ["event", "event", "event"]
|
||||
html = page(App([source]), "/s/agent/item/w1")
|
||||
assert "Recording began" in html
|
||||
assert "All 8 events" in html # the log is all still there
|
||||
|
||||
|
||||
def test_without_a_call_record_the_log_is_shown(tmp_path: Path) -> None:
|
||||
source, store = recorded(tmp_path / "a")
|
||||
store.close()
|
||||
source.calls_db.unlink()
|
||||
html = page(App([source]), "/s/agent/item/w1")
|
||||
assert "No call record" in html and "All 8 events" in html
|
||||
page(App([source]), "/s/agent/call/1", 404)
|
||||
|
||||
|
||||
def test_a_run_going_in_circles_is_flagged(tmp_path: Path) -> None:
|
||||
source = circling(tmp_path / "c")
|
||||
found = steps(source, "pr-7")
|
||||
[run] = [obj for kind, obj in found.timeline if kind == "run"]
|
||||
[circle] = run.circles
|
||||
assert (circle.kind, circle.turns) == ("repeated reply", (0, 1, 2, 3))
|
||||
assert run.turns[0].flags == ["repeated reply"] and not run.turns[4].flags
|
||||
assert "Going in circles: repeated reply at turns 1, 2, 3, 4" in page(
|
||||
App([source]), "/s/circling/item/pr-7"
|
||||
)
|
||||
assert view(source, 2).flags == ["repeated reply"]
|
||||
|
||||
|
||||
# --- one turn ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_each_message_is_shown_beside_the_log_events_it_came_from(
|
||||
agent: Source,
|
||||
) -> None:
|
||||
turn = view(agent, 2)
|
||||
assert turn.match_note is None and turn.unmatched == []
|
||||
roles = [m.role for m in turn.messages]
|
||||
assert roles == ["system", "user", "assistant", "tool"]
|
||||
# The restatement and "go" fold into one user message.
|
||||
assert [s.id for s in turn.messages[1].events] == [1, 2]
|
||||
assert [s.id for s in turn.messages[2].events] == [3]
|
||||
assert [s.id for s in turn.messages[3].events] == [4]
|
||||
assert all(s.tags == ["unchanged"] for m in turn.messages for s in m.events)
|
||||
assert calls.NOTICE in turn.messages[0].text
|
||||
|
||||
|
||||
def test_what_changed_since_the_turn_before_is_marked(agent: Source) -> None:
|
||||
first, second = view(agent, 1), view(agent, 2)
|
||||
assert not first.compared and not any(m.new for m in first.messages)
|
||||
assert second.compared
|
||||
assert [m.new for m in second.messages] == [False, False, True, True]
|
||||
assert [m.first_change for m in second.messages] == [False, False, True, False]
|
||||
assert not second.tools_changed
|
||||
assert (second.previous, second.following) == (1, 3)
|
||||
|
||||
|
||||
def test_the_turn_page_shows_the_reply_reasoning_and_raw_json(agent: Source) -> None:
|
||||
html = page(App([agent]), "/s/agent/call/1")
|
||||
assert "First, read the parser." in html # the reasoning
|
||||
assert "Request JSON, exactly as sent" in html
|
||||
assert "local-backing" in html # the model that answered
|
||||
assert "reused" not in html or "0 of 300" in html
|
||||
|
||||
|
||||
def test_how_rendering_changed_an_event_is_named() -> None:
|
||||
call = {"shortened": {"5": 1200}}
|
||||
logged = walk.Event(5, "t", "tool_result", {"result": "long"}, "w", "driver")
|
||||
sent = {"result": "lo"}
|
||||
assert walk._tags([5, "driver", "tool_result", "h"], sent, logged, call) == [
|
||||
"shortened by 1,200 characters"
|
||||
]
|
||||
told = walk.Event(6, "t", "tool_call", {"tool": "x"}, "w", "helper")
|
||||
narrated = {"text": "x → ok", "source": "view:helper"}
|
||||
assert walk._tags(
|
||||
[6, "helper", "user_message", "h"], narrated, told, {"shortened": {}}
|
||||
) == ["narrated from tool call by helper"]
|
||||
fact = walk.Event(8, "t", "fact", {"text": "agreed"}, "w", "orchestrator")
|
||||
as_said = {"text": "[orchestrator] agreed", "source": "view:orchestrator"}
|
||||
assert walk._tags(
|
||||
[8, "orchestrator", "user_message", "h"], as_said, fact, {"shortened": {}}
|
||||
) == ["labelled for this role (a fact by orchestrator)"]
|
||||
said = walk.Event(
|
||||
7, "t", "user_message", {"text": "hi", "source": "m"}, "w", "operator"
|
||||
)
|
||||
labelled = {"text": "[operator, matrix] hi", "source": "view:operator"}
|
||||
assert walk._tags(
|
||||
[7, "operator", "user_message", "h"], labelled, said, {"shortened": {}}
|
||||
) == ["labelled for this role"]
|
||||
assert walk._tags(
|
||||
[0, "harness", "fact", "h"], {"text": "x"}, None, {"shortened": {}}
|
||||
) == ["made for this call, not in the log"]
|
||||
assert walk._tags(
|
||||
[9, "driver", "compaction", "h"], {"replaces": [2, 8]}, None, {"shortened": {}}
|
||||
) == ["compaction of events 2–8"]
|
||||
|
||||
|
||||
def test_a_reply_reads_from_either_route() -> None:
|
||||
chat = {
|
||||
"choices": [
|
||||
{
|
||||
"message": {
|
||||
"content": "ok",
|
||||
"reasoning_content": "hm",
|
||||
"tool_calls": [{"function": {"name": "f", "arguments": "{}"}}],
|
||||
},
|
||||
"finish_reason": "tool_calls",
|
||||
}
|
||||
]
|
||||
}
|
||||
assert walk.reply_of(chat) == walk.Reply("ok", "hm", (("f", "{}"),), "tool_calls")
|
||||
messages = {
|
||||
"content": [
|
||||
{"type": "thinking", "thinking": "hm"},
|
||||
{"type": "text", "text": "ok"},
|
||||
{"type": "tool_use", "name": "f", "input": {"a": 1}},
|
||||
],
|
||||
"stop_reason": "tool_use",
|
||||
}
|
||||
assert walk.reply_of(messages) == walk.Reply(
|
||||
"ok", "hm", (("f", '{"a": 1}'),), "tool_use"
|
||||
)
|
||||
assert walk.reply_of("garbage") == walk.Reply()
|
||||
|
||||
|
||||
# --- untrusted text and read-only -------------------------------------------------
|
||||
|
||||
|
||||
def test_everything_shown_is_escaped_and_no_script_may_run(agent: Source) -> None:
|
||||
app = App([agent])
|
||||
for path in (
|
||||
"/s/agent/item/w1",
|
||||
"/s/agent/call/2",
|
||||
"/s/agent/call/3",
|
||||
"/s/agent/event/4",
|
||||
):
|
||||
response = app.handle(path)
|
||||
html = response.body.decode()
|
||||
assert "<script>alert" not in html
|
||||
assert "<script>alert" in html
|
||||
assert "</pre></details>" in html # can't close the page's own
|
||||
csp = response.headers["Content-Security-Policy"]
|
||||
assert "default-src 'none'" in csp and "script-src" not in csp
|
||||
assert TOKEN not in html
|
||||
|
||||
|
||||
def _digests(root: Path) -> dict[str, str]:
|
||||
"""Each file's content, except SQLite's shared-memory index, which
|
||||
readers of a live WAL database take part in by design."""
|
||||
return {
|
||||
p.name: ""
|
||||
if p.name.endswith("-shm")
|
||||
else hashlib.sha256(p.read_bytes()).hexdigest()
|
||||
for p in sorted(root.iterdir())
|
||||
if p.is_file()
|
||||
}
|
||||
|
||||
|
||||
def test_browsing_reads_only_and_leaves_no_lock_while_the_agent_writes(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
source, store = recorded(tmp_path / "a") # the store stays open, as live
|
||||
before = _digests(tmp_path / "a")
|
||||
app = App([source])
|
||||
for path in (
|
||||
"/",
|
||||
"/s/agent/",
|
||||
"/s/agent/item/w1",
|
||||
"/s/agent/item/-",
|
||||
"/s/agent/call/1",
|
||||
"/s/agent/call/2",
|
||||
"/s/agent/event/1",
|
||||
):
|
||||
page(app, path)
|
||||
# No file changed, and none was created: the closed call record has no
|
||||
# -wal or -shm, and opening it made none.
|
||||
assert _digests(tmp_path / "a") == before
|
||||
assert not Path(f"{source.calls_db}-wal").exists()
|
||||
# The agent can still write at once: nothing here holds a lock.
|
||||
store.transcript("w1", "operator").append(UserMessage(text="more", source="m:2"))
|
||||
store.close()
|
||||
assert "more" in page(app, "/s/agent/item/w1")
|
||||
|
||||
|
||||
# --- versions -----------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_store_of_an_unknown_version_is_named_not_guessed(agent: Source) -> None:
|
||||
with sqlite3.connect(agent.state_db) as conn:
|
||||
conn.execute("UPDATE meta SET value = '99' WHERE key = 'schema_version'")
|
||||
html = page(App([agent]), "/s/agent/item/w1", 409)
|
||||
assert "schema 99, not known to this page" in html
|
||||
assert "schema 99" in page(App([agent]), "/")
|
||||
|
||||
|
||||
def test_a_call_record_of_an_unknown_layout_leaves_the_log_readable(
|
||||
agent: Source,
|
||||
) -> None:
|
||||
with sqlite3.connect(agent.calls_db) as conn:
|
||||
conn.execute("UPDATE meta SET value = '2' WHERE key = 'layout_version'")
|
||||
html = page(App([agent]), "/s/agent/item/w1")
|
||||
assert "call record layout 2" in html and "All 8 events" in html
|
||||
page(App([agent]), "/s/agent/call/1", 409)
|
||||
|
||||
|
||||
def test_item_data_that_doesnt_parse_still_renders(agent: Source) -> None:
|
||||
with sqlite3.connect(agent.state_db) as conn:
|
||||
conn.execute("UPDATE work_items SET data = 'not json', position = 'NEW_STATE'")
|
||||
html = page(App([agent]), "/s/agent/item/w1")
|
||||
assert "not json" in html and "NEW_STATE" in html
|
||||
assert "NEW_STATE" in page(App([agent]), "/s/agent/")
|
||||
|
||||
|
||||
def test_unknown_paths_are_not_found(agent: Source) -> None:
|
||||
app = App([agent])
|
||||
page(app, "/s/nobody/", 404)
|
||||
page(app, "/s/agent/item/nope", 404)
|
||||
page(app, "/s/agent/call/999", 404)
|
||||
page(app, "/s/agent/event/999", 404)
|
||||
page(app, "/s/agent/call/x", 404)
|
||||
page(app, "/etc/passwd", 404)
|
||||
assert app.handle("/static/style.css").content_type.startswith("text/css")
|
||||
@@ -0,0 +1,149 @@
|
||||
"""Stores written by the agents' own code, for the page to read (X6): a real
|
||||
state store and a real call record from a driver run against a fake
|
||||
LiteLLM, so a change in nisse-core that breaks the page fails here."""
|
||||
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_core.log.canonical import Fact, UserMessage
|
||||
from nisse_core.loop.driver import Driver
|
||||
from nisse_core.loop.tools import tool
|
||||
from nisse_core.models import calls
|
||||
from nisse_core.models.context import BudgetedClient
|
||||
from nisse_core.models.llm import LiteLLMClient
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state.store import StateStore
|
||||
from nisse_ui.sources import Source
|
||||
|
||||
TOKEN = "sk-secret-litellm-token"
|
||||
HOSTILE = "<script>alert('x')</script></pre></details>"
|
||||
READ = tool("read_file", "read a file", path="the file")
|
||||
POST = tool("post_comment", "post a comment", body="the comment")
|
||||
FINISH = tool("finish", "finish the work", summary="what was done")
|
||||
|
||||
|
||||
def reply(
|
||||
content: str = "",
|
||||
*,
|
||||
call: tuple[str, dict[str, Any]] | None = None,
|
||||
reasoning: str = "",
|
||||
timings: dict[str, int] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
message: dict[str, Any] = {"role": "assistant", "content": content}
|
||||
if reasoning:
|
||||
message["reasoning_content"] = reasoning
|
||||
if call:
|
||||
message["tool_calls"] = [
|
||||
{
|
||||
"id": f"x{abs(hash(json.dumps(call)))}",
|
||||
"type": "function",
|
||||
"function": {"name": call[0], "arguments": json.dumps(call[1])},
|
||||
}
|
||||
]
|
||||
data: dict[str, Any] = {
|
||||
"model": "local-backing",
|
||||
"choices": [{"message": message, "finish_reason": "stop"}],
|
||||
"usage": {"prompt_tokens": 120, "completion_tokens": 7},
|
||||
}
|
||||
if timings is not None:
|
||||
data["timings"] = timings
|
||||
return data
|
||||
|
||||
|
||||
class Server:
|
||||
"""A fake LiteLLM answering each request in turn."""
|
||||
|
||||
def __init__(self, *answers: dict[str, Any]) -> None:
|
||||
self.answers = list(answers)
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
data = self.answers.pop(0) if self.answers else reply("ok")
|
||||
return httpx.Response(200, json=data)
|
||||
|
||||
|
||||
def client(server: Server, recorder: calls.CallRecorder) -> BudgetedClient:
|
||||
inner = LiteLLMClient(
|
||||
"https://litellm.test", TOKEN, transport=httpx.MockTransport(server)
|
||||
)
|
||||
return BudgetedClient(inner, lambda m: (32768, 1024), call_recorder=recorder)
|
||||
|
||||
|
||||
def store_at(root: Path, store_class: type[StateStore] = StateStore) -> StateStore:
|
||||
return store_class(root / "state.db", oplog=OpLog("test", io.StringIO()))
|
||||
|
||||
|
||||
def recorded(root: Path) -> tuple[Source, StateStore]:
|
||||
"""One work item, one driver run of three turns: a tool call whose
|
||||
result is hostile text, a prose reply the driver corrects, then the
|
||||
finish. And one agent-level call. The store stays open, as a live
|
||||
agent's does."""
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
recorder = calls.CallRecorder(calls.path_for(root / "state.db"))
|
||||
store = store_at(root)
|
||||
store.open_work_item("w1", title="Fix the parser", position="RUNNING", cause="t")
|
||||
store.transcript("w1", "orchestrator").append(
|
||||
Fact(text="The operator agreed to this restatement: fix the parser.")
|
||||
)
|
||||
store.transcript("w1", "operator").append(UserMessage(text="go", source="m:1"))
|
||||
server = Server(
|
||||
reply(
|
||||
call=("read_file", {"path": "parser.py"}),
|
||||
reasoning="First, read the parser.",
|
||||
timings={"cache_n": 0, "prompt_n": 300},
|
||||
),
|
||||
reply(
|
||||
"I believe the parser is fine now.",
|
||||
timings={"cache_n": 280, "prompt_n": 60},
|
||||
),
|
||||
reply(
|
||||
call=("finish", {"summary": "fixed"}),
|
||||
timings={"cache_n": 330, "prompt_n": 20},
|
||||
),
|
||||
)
|
||||
Driver(client(server, recorder), "local").run(
|
||||
store.transcript("w1", "driver"),
|
||||
system="You are the driver.",
|
||||
tools=[READ, FINISH],
|
||||
waiting=("finish",),
|
||||
workspace=None,
|
||||
handlers={"read_file": lambda args: (f"contents {HOSTILE}", True)},
|
||||
)
|
||||
store.transcript(None, "operator").append(UserMessage(text="hello", source="m:9"))
|
||||
with calls.run(recorder, work_item=None, role="classifier", kind="classification"):
|
||||
client(Server(reply("harmless")), recorder).complete(
|
||||
store.transcript(None, "operator").read(), None, "local", system="Classify."
|
||||
)
|
||||
recorder.close()
|
||||
return Source("agent", root / "state.db", "agent"), store
|
||||
|
||||
|
||||
def circling(root: Path) -> Source:
|
||||
"""A run that posts the same comment over and over, as Flash-Next did
|
||||
on 2026-10-24, until the finish."""
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
recorder = calls.CallRecorder(calls.path_for(root / "state.db"))
|
||||
store = store_at(root)
|
||||
store.open_work_item("pr-7", title="Review #7", position="RUNNING", cause="t")
|
||||
store.transcript("pr-7", "operator").append(UserMessage(text="review", source="m"))
|
||||
answers = [
|
||||
reply(call=("post_comment", {"body": f"Duplicate, please disregard ({n})."}))
|
||||
for n in range(4)
|
||||
]
|
||||
Driver(
|
||||
client(Server(*answers, reply(call=("finish", {"summary": "x"}))), recorder),
|
||||
"local",
|
||||
).run(
|
||||
store.transcript("pr-7", "driver"),
|
||||
system="You review.",
|
||||
tools=[POST, FINISH],
|
||||
waiting=("finish",),
|
||||
workspace=None,
|
||||
handlers={"post_comment": lambda args: ("posted", True)},
|
||||
)
|
||||
recorder.close()
|
||||
store.close()
|
||||
return Source("circling", root / "state.db", "agent")
|
||||
Reference in new issue
Block a user