Merge pull request 'M7.1: extract nisse-core' (#143) from extract-core into main
Reviewed-on: #143 Reviewed-by: Don Garrett <dgarrett@acm.org>
This commit was merged in pull request #143.
This commit is contained in:
commit
5ea6e4610f
119 files changed
+1086
-912
No files matched your search
@@ -59,7 +59,7 @@ jobs:
|
||||
|
||||
- name: Install sshd
|
||||
# The tests reach "the build VM" over real SSH, through a throwaway
|
||||
# sshd the test fixture starts (nisse_dev/testing.py). Skipped when the image has it; otherwise from
|
||||
# sshd the test fixture starts (nisse_core/testing.py). Skipped when the image has it; otherwise from
|
||||
# a kernel.org mirror: archive.ubuntu.com has been slow enough to add
|
||||
# minutes to every job.
|
||||
run: scripts/ci-sshd.sh
|
||||
|
||||
+2
-2
@@ -5,8 +5,8 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.testing import local_sshd
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.testing import local_sshd
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
|
||||
@@ -131,7 +131,13 @@ naming a milestone (D4).
|
||||
|
||||
## Part L — The shared library (`nisse-core`)
|
||||
|
||||
Everything in this part is used by every agent.
|
||||
Everything in this part is used by every agent. In code it is the `nisse_core`
|
||||
package: the transcript (L7) and its renderers, the LLM client and model catalog
|
||||
(L9), context fitting and per-role views (L6), the state store and reconciliation
|
||||
(L8), transports (L5), SSH execution (L13), the vault and memory (L10, L6.4),
|
||||
logging and secrets loading (L12), and the Gitea client (L1). An agent extends
|
||||
the store with its own tables, roles and views; the loop is still `nisse-dev`'s
|
||||
until a second agent shares it.
|
||||
|
||||
### L1. The work model
|
||||
|
||||
|
||||
@@ -535,6 +535,11 @@ Authelia.
|
||||
|
||||
### Phase 6 — Production
|
||||
|
||||
*Waiting on infrastructure (2026-10-13):* the agent host doesn't exist yet, and
|
||||
Terraform isn't running; the operator is planning both, Terraform with
|
||||
`nisse-ops`. Until then M5.4 and M6.1–M6.4 wait, and the build VMs stay made by
|
||||
hand (Ubuntu 24 now, 26 soon), with `vm ready` for a rebuilt one's host key.
|
||||
|
||||
**M6.1 The agent host.** *Needs discussion.* Containers, volumes, snapshots,
|
||||
per-agent networks (X1, X2, X4).
|
||||
|
||||
@@ -550,7 +555,26 @@ LLDAP principal (L12).
|
||||
|
||||
### Phase 7 — `nisse-core` and `nisse-ops`
|
||||
|
||||
**M7.1 Extract `nisse-core`.** **M7.2 MCP servers** (#37). **M7.3 Approved step
|
||||
**M7.1 Extract `nisse-core`.** *Done.* What isn't the dev agent's moved to
|
||||
`nisse-core`, unchanged in behaviour, with its tests: the transcript and its
|
||||
renderers, the LLM client and model catalog, context fitting and views, the state
|
||||
store, reconciliation, the worker and `stop`, transports (Matrix), SSH execution,
|
||||
the vault and memory, logging, `.env` loading, and the Gitea client (whole: L1
|
||||
makes Gitea every agent's tracker). `nisse-core` never imports an agent (a test
|
||||
says so).
|
||||
|
||||
- `nisse_core.state.StateStore` holds the tables every agent has; an agent's
|
||||
subclass adds its own tables (with their schema version and migrations), its
|
||||
model roles, and what each role is shown of the others (`views`).
|
||||
`nisse_dev.state.StateStore` adds the build VM blocks (D3); stores already at
|
||||
version 5 open unchanged.
|
||||
- The vault names whose writes it records (`agent`), rather than assuming one.
|
||||
- Deferred until `nisse-ops` has a loop to share: splitting the loop (`loop/*`)
|
||||
and configuration into a common part and the dev agent's. The dev agent's
|
||||
`git`, delegation, `ask_expert`, executor tools and evaluation capture stay
|
||||
in `nisse-dev`.
|
||||
|
||||
**M7.2 MCP servers** (#37). **M7.3 Approved step
|
||||
sequences** (O2). **M7.4 Ops intake and duties** from the infrastructure
|
||||
repository (O5). **M7.5 The ops tool VM and deployment** (O6). **M7.6 Requests
|
||||
between agents:** `nisse-dev` asking for build VMs (L1, O5). All *needs discussion*
|
||||
|
||||
+18
-7
@@ -1,11 +1,22 @@
|
||||
# nisse-core
|
||||
|
||||
The shared library both Nisse agents sit on (Part L of the architecture).
|
||||
The shared library the Nisse agents sit on (Part L of the architecture). It
|
||||
never imports an agent.
|
||||
|
||||
Most of it still lives in `nisse-dev`, to be extracted as `nisse-ops` arrives.
|
||||
What is here so far:
|
||||
- **Transcript and models:** `canonical` (the log's events, L7), `render` and
|
||||
`render_anthropic`, `llm` (the one client, L9), `catalog` and `probe` (what
|
||||
the models are), `context` (fitting a request to its window, L6), `views`
|
||||
(what each role is shown of the others).
|
||||
- **Durability:** `state` (the store, L8; an agent subclasses it for its own
|
||||
tables, roles and views), `reconcile`, `worker`, `cancel`, `dump`.
|
||||
- **Conversation:** `transport`, `matrix` (L5).
|
||||
- **Execution:** `execution` (SSH to a tool VM, L13); `testing` (a throwaway
|
||||
sshd for tests).
|
||||
- **Knowledge:** `vault` (L10), `memory` (L6.4).
|
||||
- **Operations:** `oplog`, `envfile` (L12); `gitea` (the tracker and forge, L1).
|
||||
- **Authority:** `actions` (P3, P4, L4): `Action`, `Verdict`, and static
|
||||
reading of shell commands, which each agent's fast-path rules build on
|
||||
(`nisse-dev`'s are in `nisse_dev.loop.actions`).
|
||||
|
||||
- `nisse_core.actions`: actions by their effects (P3, P4, L4). The `Action`
|
||||
and `Verdict` types, and static reading of shell commands, which each
|
||||
agent's fast-path rules build on (`nisse-dev`'s are in
|
||||
`nisse_dev.loop.actions`).
|
||||
The loop is still `nisse-dev`'s, until `nisse-ops` has one to share it with
|
||||
(Development Plan M7.1).
|
||||
@@ -3,7 +3,10 @@ name = "nisse-core"
|
||||
version = "0.1.0"
|
||||
description = "Shared agent library for nisse-dev and nisse-ops"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = []
|
||||
dependencies = [
|
||||
"httpx>=0.27",
|
||||
"pyyaml>=6",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["uv_build>=0.12,<0.13"]
|
||||
|
||||
@@ -8,8 +8,8 @@ record of a stop is written by the worker when the stopped work returns.
|
||||
|
||||
import threading
|
||||
|
||||
from nisse_dev.canonical import ToolCall, ToolResult
|
||||
from nisse_dev.state import Transcript
|
||||
from nisse_core.canonical import ToolCall, ToolResult
|
||||
from nisse_core.state import Transcript
|
||||
|
||||
STOPPED_RESULT = "not run: the operator stopped the agent"
|
||||
|
||||
File renamed without changes.
@@ -19,7 +19,7 @@ from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.llm import LLMClient, LLMError
|
||||
from nisse_core.llm import LLMClient, LLMError
|
||||
|
||||
# LiteLLM's provider name for models we have a vendor-native adapter for.
|
||||
ANTHROPIC_PROVIDER = "anthropic"
|
||||
@@ -28,7 +28,7 @@ from collections.abc import Callable, Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
DelegationStarted,
|
||||
@@ -38,7 +38,7 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.llm import DEFAULT_MAX_OUTPUT_TOKENS, LLMError
|
||||
from nisse_core.llm import DEFAULT_MAX_OUTPUT_TOKENS, LLMError
|
||||
|
||||
# English and code run at about four characters a token. Three is deliberately
|
||||
# pessimistic, before any correction from what the server reports.
|
||||
@@ -1,6 +1,6 @@
|
||||
"""A readable dump of a state store, for bring-up runs and debugging.
|
||||
|
||||
python -m nisse_dev.dump STATE_DB [--full]
|
||||
python -m nisse_core.dump STATE_DB [--full]
|
||||
|
||||
Opens the store read-only (SQLite `mode=ro`), so it is safe to run against a
|
||||
live agent's database: it never migrates, writes or locks it. Prints the work
|
||||
File renamed without changes.
File renamed without changes.
@@ -16,8 +16,8 @@ from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.reconcile import Reconciler
|
||||
from nisse_dev.state import Intent
|
||||
from nisse_core.reconcile import Reconciler
|
||||
from nisse_core.state import Intent
|
||||
|
||||
PR_PAGE_SIZE = 50
|
||||
MAX_PR_PAGES = 40 # 2000 PRs; beyond that, say so rather than guess
|
||||
@@ -17,15 +17,15 @@ from typing import Any, ClassVar, Protocol
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
StoredEvent,
|
||||
ToolCall,
|
||||
mint_call_id,
|
||||
)
|
||||
from nisse_dev.render import OpenAIChatRenderer
|
||||
from nisse_dev.render_anthropic import AnthropicMessagesRenderer
|
||||
from nisse_core.render import OpenAIChatRenderer
|
||||
from nisse_core.render_anthropic import AnthropicMessagesRenderer
|
||||
|
||||
|
||||
class LLMError(RuntimeError):
|
||||
@@ -27,7 +27,7 @@ from urllib.parse import unquote
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.transport import InboundMessage
|
||||
from nisse_core.transport import InboundMessage
|
||||
|
||||
# A mention pill at the start of an HTML body: the link holds the user ID,
|
||||
# the link text is what the plain body starts with.
|
||||
@@ -14,7 +14,7 @@ from dataclasses import dataclass
|
||||
|
||||
import yaml
|
||||
|
||||
from nisse_dev.vault import Vault, VaultError, split_frontmatter
|
||||
from nisse_core.vault import Vault, VaultError, split_frontmatter
|
||||
|
||||
INDEX = "index" # the note the harness keeps; never the model's
|
||||
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,63}")
|
||||
File renamed without changes.
@@ -24,7 +24,7 @@ from dataclasses import dataclass
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.catalog import ModelFacts
|
||||
from nisse_core.catalog import ModelFacts
|
||||
|
||||
# Of a probe's size, how much must come back for it to count as uncut. A
|
||||
# chat template adds a little, and a tokenizer counts a little differently.
|
||||
@@ -11,7 +11,7 @@ from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from nisse_dev.state import Intent, StateStore
|
||||
from nisse_core.state import Intent, StateStore
|
||||
|
||||
# Returns True if the intent's external effect verifiably happened (do not
|
||||
# repeat it), False if it verifiably never started (safe to discard), or
|
||||
@@ -11,7 +11,7 @@ this skeleton.
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
Compaction,
|
||||
DelegationCompleted,
|
||||
+2
-2
@@ -18,7 +18,7 @@ route and this renderer builds the vendor-native body.
|
||||
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
Compaction,
|
||||
DelegationCompleted,
|
||||
@@ -31,7 +31,7 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.render import effective_events
|
||||
from nisse_core.render import effective_events
|
||||
|
||||
DEFAULT_MAX_TOKENS = 4096
|
||||
NO_RESULT_PLACEHOLDER = "[no result recorded]"
|
||||
@@ -0,0 +1,576 @@
|
||||
"""Durable state store (architecture L8).
|
||||
|
||||
SQLite in WAL mode with ``synchronous=FULL``: every commit is fsynced, so a
|
||||
power cut mid-write can never produce torn state. Every state transition and
|
||||
intent-log write is a single immediately-committed statement — in-memory state
|
||||
that hasn't hit disk is assumed lost.
|
||||
|
||||
The intent log records a unit of work *before* it starts, so a restart knows
|
||||
what was in flight (see ``nisse_core.reconcile`` for how pending intents are
|
||||
resolved against external truth on boot).
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Self
|
||||
|
||||
from nisse_core.canonical import (
|
||||
CanonicalEvent,
|
||||
StoredEvent,
|
||||
UserMessage,
|
||||
event_type,
|
||||
from_payload,
|
||||
to_payload,
|
||||
)
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.views import project
|
||||
|
||||
# The tables every agent has. An agent's own tables come on top: a subclass
|
||||
# names them, the version they bring, and the migrations to it.
|
||||
CURRENT_SCHEMA_VERSION = 4
|
||||
|
||||
# Stores older than this predate work items (Development Plan M1.1). There was
|
||||
# no production data to carry over, so there is no migration from them: start
|
||||
# a fresh store (scripts/bringup.sh start sets the old one aside).
|
||||
OLDEST_SUPPORTED_VERSION = 4
|
||||
|
||||
Migration = Callable[[sqlite3.Connection], None]
|
||||
|
||||
# Who produced an event (L7): the operator, the orchestrator's own code, or a
|
||||
# model role (L9). An agent adds its own roles.
|
||||
ACTORS = frozenset(
|
||||
{"operator", "orchestrator", "driver", "helper", "expert", "classifier"}
|
||||
)
|
||||
|
||||
|
||||
class StateSchemaError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkItem:
|
||||
id: str
|
||||
scope: str | None
|
||||
title: str
|
||||
status: str # open | closed
|
||||
outcome: str | None # completed | abandoned, once closed
|
||||
position: str # where the loop is for this item
|
||||
data: dict[str, Any]
|
||||
opened_at: str
|
||||
updated_at: str
|
||||
closed_at: str | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Intent:
|
||||
id: int
|
||||
kind: str
|
||||
payload: dict[str, Any]
|
||||
status: str
|
||||
created_at: str
|
||||
resolved_at: str | None
|
||||
work_item: str | None = None
|
||||
|
||||
|
||||
_SCHEMA = [
|
||||
"CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)",
|
||||
"""
|
||||
CREATE TABLE work_items (
|
||||
id TEXT PRIMARY KEY,
|
||||
scope TEXT,
|
||||
title TEXT NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('open', 'closed')),
|
||||
outcome TEXT,
|
||||
position TEXT NOT NULL,
|
||||
data TEXT NOT NULL DEFAULT '{}',
|
||||
opened_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
closed_at TEXT
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE agent (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
active_item TEXT REFERENCES work_items(id),
|
||||
idle_from_event INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE intent_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
kind TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
created_at TEXT NOT NULL,
|
||||
resolved_at TEXT,
|
||||
work_item TEXT
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE canonical_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ts TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
work_item TEXT,
|
||||
actor TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
"CREATE INDEX canonical_events_by_item ON canonical_events (work_item, id)",
|
||||
"""
|
||||
CREATE TABLE eval_outcomes (
|
||||
event_id INTEGER PRIMARY KEY,
|
||||
verdict TEXT NOT NULL,
|
||||
note TEXT NOT NULL DEFAULT '',
|
||||
marked_at TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
]
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(UTC).isoformat()
|
||||
|
||||
|
||||
class StateStore:
|
||||
"""The store every agent shares. An agent's subclass adds its own tables
|
||||
(``SCHEMA``, with the version they bring and ``MIGRATIONS`` to it), its
|
||||
model roles (``ACTORS``) and what each role is shown of the others
|
||||
(``SHOWN``, see ``views``)."""
|
||||
|
||||
AGENT = "nisse" # what the operational log names
|
||||
SCHEMA_VERSION = CURRENT_SCHEMA_VERSION
|
||||
SCHEMA: tuple[str, ...] = ()
|
||||
# Maps version N to the migration that takes the schema from N to N+1.
|
||||
MIGRATIONS: Mapping[int, Migration] = {}
|
||||
ACTORS: frozenset[str] = ACTORS
|
||||
SHOWN: Mapping[str, Mapping[str, str]] = {}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
path: str | Path,
|
||||
*,
|
||||
schema_version: int | None = None,
|
||||
migrations: Mapping[int, Migration] | None = None,
|
||||
oplog: OpLog | None = None,
|
||||
) -> None:
|
||||
# Log emission is owned by the store (L12): the same code path that
|
||||
# performs a durable write emits its log line, right after commit.
|
||||
self.oplog = oplog if oplog is not None else OpLog(self.AGENT)
|
||||
self._conn = sqlite3.connect(path, isolation_level=None)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA synchronous=FULL")
|
||||
self._conn.execute("PRAGMA busy_timeout=5000")
|
||||
self._target_version = (
|
||||
self.SCHEMA_VERSION if schema_version is None else schema_version
|
||||
)
|
||||
self.closed = False
|
||||
self._init_or_migrate(self.MIGRATIONS if migrations is None else migrations)
|
||||
|
||||
def _init_or_migrate(self, migrations: Mapping[int, Migration]) -> None:
|
||||
fresh = (
|
||||
self._conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name='meta'"
|
||||
).fetchone()
|
||||
is None
|
||||
)
|
||||
if fresh:
|
||||
self._in_transaction(self._create_schema)
|
||||
return
|
||||
stored = self.schema_version
|
||||
if stored < OLDEST_SUPPORTED_VERSION <= self._target_version:
|
||||
raise StateSchemaError(
|
||||
f"state store schema version {stored} predates work items "
|
||||
f"(version {OLDEST_SUPPORTED_VERSION}); start a fresh store "
|
||||
"(scripts/bringup.sh start sets the old one aside)"
|
||||
)
|
||||
if stored > self._target_version:
|
||||
raise StateSchemaError(
|
||||
f"state schema version {stored} is newer than this code "
|
||||
f"supports ({self._target_version}); refusing to open"
|
||||
)
|
||||
for version in range(stored, self._target_version):
|
||||
migrate = migrations.get(version)
|
||||
if migrate is None:
|
||||
raise StateSchemaError(
|
||||
f"no migration registered from schema version {version}"
|
||||
)
|
||||
self._in_transaction(lambda m=migrate, v=version: self._apply(m, v))
|
||||
self.oplog.emit(
|
||||
"schema.migrate",
|
||||
cause="startup",
|
||||
from_version=version,
|
||||
to_version=version + 1,
|
||||
)
|
||||
|
||||
def _create_schema(self) -> None:
|
||||
for statement in (*_SCHEMA, *self.SCHEMA):
|
||||
self._conn.execute(statement)
|
||||
self._conn.execute(
|
||||
"INSERT INTO meta (key, value) VALUES ('schema_version', ?)",
|
||||
(str(self._target_version),),
|
||||
)
|
||||
|
||||
def _apply(self, migrate: Migration, from_version: int) -> None:
|
||||
migrate(self._conn)
|
||||
self._conn.execute(
|
||||
"UPDATE meta SET value = ? WHERE key = 'schema_version'",
|
||||
(str(from_version + 1),),
|
||||
)
|
||||
|
||||
def _in_transaction(self, work: Callable[[], None]) -> None:
|
||||
self._conn.execute("BEGIN IMMEDIATE")
|
||||
try:
|
||||
work()
|
||||
except BaseException:
|
||||
self._conn.execute("ROLLBACK")
|
||||
raise
|
||||
self._conn.execute("COMMIT")
|
||||
|
||||
@property
|
||||
def schema_version(self) -> int:
|
||||
row = self._conn.execute(
|
||||
"SELECT value FROM meta WHERE key = 'schema_version'"
|
||||
).fetchone()
|
||||
return int(row["value"])
|
||||
|
||||
# --- work items (L1) ---------------------------------------------------
|
||||
|
||||
def _ensure_agent_row(self) -> None:
|
||||
self._conn.execute(
|
||||
"INSERT INTO agent (id, active_item, updated_at) VALUES (1, NULL, ?) "
|
||||
"ON CONFLICT(id) DO NOTHING",
|
||||
(utc_now(),),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _item(row: sqlite3.Row) -> WorkItem:
|
||||
return WorkItem(
|
||||
id=row["id"],
|
||||
scope=row["scope"],
|
||||
title=row["title"],
|
||||
status=row["status"],
|
||||
outcome=row["outcome"],
|
||||
position=row["position"],
|
||||
data=json.loads(row["data"]),
|
||||
opened_at=row["opened_at"],
|
||||
updated_at=row["updated_at"],
|
||||
closed_at=row["closed_at"],
|
||||
)
|
||||
|
||||
def work_item(self, item_id: str) -> WorkItem | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM work_items WHERE id = ?", (item_id,)
|
||||
).fetchone()
|
||||
return None if row is None else self._item(row)
|
||||
|
||||
def work_items(self, status: str | None = None) -> list[WorkItem]:
|
||||
query = "SELECT * FROM work_items"
|
||||
args: tuple[Any, ...] = ()
|
||||
if status is not None:
|
||||
query += " WHERE status = ?"
|
||||
args = (status,)
|
||||
rows = self._conn.execute(query + " ORDER BY opened_at", args).fetchall()
|
||||
return [self._item(row) for row in rows]
|
||||
|
||||
def active_item(self) -> WorkItem | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT w.* FROM agent a JOIN work_items w ON w.id = a.active_item "
|
||||
"WHERE a.id = 1"
|
||||
).fetchone()
|
||||
return None if row is None else self._item(row)
|
||||
|
||||
def open_work_item(
|
||||
self,
|
||||
item_id: str,
|
||||
*,
|
||||
title: str,
|
||||
scope: str | None = None,
|
||||
position: str,
|
||||
data: dict[str, Any] | None = None,
|
||||
cause: str,
|
||||
) -> WorkItem:
|
||||
now = utc_now()
|
||||
self._conn.execute(
|
||||
"INSERT INTO work_items (id, scope, title, status, position, data, "
|
||||
"opened_at, updated_at) VALUES (?, ?, ?, 'open', ?, ?, ?, ?)",
|
||||
(item_id, scope, title, position, json.dumps(data or {}), now, now),
|
||||
)
|
||||
self.oplog.emit("work_item.open", cause=cause, work_item=item_id, scope=scope)
|
||||
item = self.work_item(item_id)
|
||||
assert item is not None
|
||||
return item
|
||||
|
||||
def set_active(self, item_id: str | None, *, cause: str) -> None:
|
||||
"""Point the agent at a work item, or at none (idle). Going idle
|
||||
records where in the log idleness began, so messages sent while idle
|
||||
can be carried into the next work item (carry_idle_messages)."""
|
||||
self._ensure_agent_row()
|
||||
previous = self._conn.execute(
|
||||
"SELECT active_item FROM agent WHERE id = 1"
|
||||
).fetchone()["active_item"]
|
||||
if item_id is None:
|
||||
last = self._conn.execute(
|
||||
"SELECT COALESCE(MAX(id), 0) AS last FROM canonical_events"
|
||||
).fetchone()["last"]
|
||||
self._conn.execute(
|
||||
"UPDATE agent SET active_item = NULL, idle_from_event = ?, "
|
||||
"updated_at = ? WHERE id = 1",
|
||||
(last, utc_now()),
|
||||
)
|
||||
else:
|
||||
self._conn.execute(
|
||||
"UPDATE agent SET active_item = ?, updated_at = ? WHERE id = 1",
|
||||
(item_id, utc_now()),
|
||||
)
|
||||
if previous != item_id:
|
||||
self.oplog.emit(
|
||||
"work_item.switch",
|
||||
cause=cause,
|
||||
**{"from": previous, "to": item_id},
|
||||
)
|
||||
|
||||
def update_work_item(
|
||||
self,
|
||||
item_id: str,
|
||||
*,
|
||||
position: str,
|
||||
data: dict[str, Any],
|
||||
cause: str,
|
||||
) -> None:
|
||||
"""Record a work item's new position and data; the full data is
|
||||
passed every time (no partial updates)."""
|
||||
previous = self.work_item(item_id)
|
||||
if previous is None:
|
||||
raise KeyError(f"no work item {item_id!r}")
|
||||
self._conn.execute(
|
||||
"UPDATE work_items SET position = ?, data = ?, updated_at = ? WHERE id = ?",
|
||||
(position, json.dumps(data), utc_now(), item_id),
|
||||
)
|
||||
self.oplog.emit(
|
||||
"state.transition",
|
||||
cause=cause,
|
||||
**{"from": previous.position, "to": position, "project": previous.scope},
|
||||
)
|
||||
|
||||
def close_work_item(self, item_id: str, *, outcome: str, cause: str) -> None:
|
||||
now = utc_now()
|
||||
row = self._conn.execute(
|
||||
"UPDATE work_items SET status = 'closed', outcome = ?, closed_at = ?, "
|
||||
"updated_at = ? WHERE id = ? AND status = 'open' RETURNING id",
|
||||
(outcome, now, now, item_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(f"no open work item {item_id!r}")
|
||||
self.oplog.emit(
|
||||
"work_item.close", cause=cause, work_item=item_id, outcome=outcome
|
||||
)
|
||||
|
||||
def carry_idle_messages(self, item_id: str) -> int:
|
||||
"""Copy what the operator said while the agent was idle into a work
|
||||
item that has just opened (D1: messages sent before `use` are kept
|
||||
for it). Copied as new events, with the original's ID in the source,
|
||||
never by reassigning the originals: the log is never edited (L7)."""
|
||||
self._ensure_agent_row()
|
||||
since = self._conn.execute(
|
||||
"SELECT idle_from_event FROM agent WHERE id = 1"
|
||||
).fetchone()["idle_from_event"]
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item IS NULL "
|
||||
"AND actor = 'operator' AND type = 'user_message' AND id > ? ORDER BY id",
|
||||
(since,),
|
||||
).fetchall()
|
||||
log = self.transcript(item_id, "operator")
|
||||
for row in rows:
|
||||
payload = json.loads(row["payload"])
|
||||
log.append(
|
||||
UserMessage(
|
||||
text=payload["text"],
|
||||
source=f"{payload.get('source', '')} (sent while idle, #{row['id']})",
|
||||
)
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
# --- intents (L8) ------------------------------------------------------
|
||||
|
||||
def begin_intent(
|
||||
self,
|
||||
kind: str,
|
||||
payload: dict[str, Any],
|
||||
*,
|
||||
cause: str,
|
||||
work_item: str | None = None,
|
||||
) -> int:
|
||||
"""Durably record that a unit of work is about to start, owned by
|
||||
the work item it belongs to (L1)."""
|
||||
cursor = self._conn.execute(
|
||||
"INSERT INTO intent_log (kind, payload, created_at, work_item) "
|
||||
"VALUES (?, ?, ?, ?)",
|
||||
(kind, json.dumps(payload), utc_now(), work_item),
|
||||
)
|
||||
assert cursor.lastrowid is not None
|
||||
self.oplog.emit(
|
||||
"intent.begin", cause=cause, intent_id=cursor.lastrowid, kind=kind
|
||||
)
|
||||
return cursor.lastrowid
|
||||
|
||||
def commit_intent(self, intent_id: int, *, cause: str) -> None:
|
||||
self._resolve_intent(intent_id, "committed", "intent.commit", cause)
|
||||
|
||||
def abort_intent(self, intent_id: int, *, cause: str) -> None:
|
||||
self._resolve_intent(intent_id, "aborted", "intent.abort", cause)
|
||||
|
||||
def _resolve_intent(
|
||||
self, intent_id: int, status: str, event: str, cause: str
|
||||
) -> None:
|
||||
row = self._conn.execute(
|
||||
"UPDATE intent_log SET status = ?, resolved_at = ? "
|
||||
"WHERE id = ? AND status = 'pending' RETURNING kind",
|
||||
(status, utc_now(), intent_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(f"no pending intent with id {intent_id}")
|
||||
self.oplog.emit(event, cause=cause, intent_id=intent_id, kind=row["kind"])
|
||||
|
||||
def pending_intents(self) -> list[Intent]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM intent_log WHERE status = 'pending' ORDER BY id"
|
||||
).fetchall()
|
||||
return [
|
||||
Intent(
|
||||
id=row["id"],
|
||||
kind=row["kind"],
|
||||
payload=json.loads(row["payload"]),
|
||||
status=row["status"],
|
||||
created_at=row["created_at"],
|
||||
resolved_at=row["resolved_at"],
|
||||
work_item=row["work_item"],
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
||||
# --- the canonical log (L7) -------------------------------------------
|
||||
|
||||
def transcript(self, work_item: str | None, actor: str) -> "Transcript":
|
||||
"""The log as one role sees it for one work item: what it reads is
|
||||
that item's events only, and what it writes carries the item and the
|
||||
actor. None is the agent-level scope (L6)."""
|
||||
if actor not in self.ACTORS:
|
||||
raise ValueError(f"unknown actor {actor!r}")
|
||||
return Transcript(self, work_item, actor)
|
||||
|
||||
def all_events(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""Every event across work items, for the few readers whose job
|
||||
spans them (evaluation capture, dumps). Model calls never use this."""
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE id > ? ORDER BY id", (after,)
|
||||
).fetchall()
|
||||
return [_stored(row) for row in rows]
|
||||
|
||||
def _append(self, event: CanonicalEvent, work_item: str | None, actor: str) -> int:
|
||||
cursor = self._conn.execute(
|
||||
"INSERT INTO canonical_events (ts, type, payload, work_item, actor) "
|
||||
"VALUES (?, ?, ?, ?, ?)",
|
||||
(
|
||||
utc_now(),
|
||||
event_type(event),
|
||||
json.dumps(to_payload(event)),
|
||||
work_item,
|
||||
actor,
|
||||
),
|
||||
)
|
||||
assert cursor.lastrowid is not None
|
||||
return cursor.lastrowid
|
||||
|
||||
def _read(self, work_item: str | None, after: int) -> list[StoredEvent]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item IS ? AND id > ? "
|
||||
"ORDER BY id",
|
||||
(work_item, after),
|
||||
).fetchall()
|
||||
return [_stored(row) for row in rows]
|
||||
|
||||
def mark_outcome(self, event_id: int, verdict: str, note: str = "") -> None:
|
||||
"""Record whether a delegation or consult was good (L9).
|
||||
|
||||
Kept out of the canonical log on purpose: the log is an append-only
|
||||
record of what happened, and a verdict is a later judgement about it
|
||||
that may be revised.
|
||||
"""
|
||||
self._conn.execute(
|
||||
"INSERT INTO eval_outcomes (event_id, verdict, note, marked_at) "
|
||||
"VALUES (?, ?, ?, ?) ON CONFLICT(event_id) DO UPDATE SET "
|
||||
"verdict = excluded.verdict, note = excluded.note, "
|
||||
"marked_at = excluded.marked_at",
|
||||
(event_id, verdict, note, utc_now()),
|
||||
)
|
||||
|
||||
def outcomes(self) -> dict[int, tuple[str, str]]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT event_id, verdict, note FROM eval_outcomes"
|
||||
).fetchall()
|
||||
return {r["event_id"]: (r["verdict"], r["note"]) for r in rows}
|
||||
|
||||
def close(self) -> None:
|
||||
self._conn.close()
|
||||
self.closed = True
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: object) -> None:
|
||||
self.close()
|
||||
|
||||
|
||||
def _stored(row: sqlite3.Row) -> StoredEvent:
|
||||
return StoredEvent(
|
||||
id=row["id"],
|
||||
ts=row["ts"],
|
||||
event=from_payload(row["type"], json.loads(row["payload"])),
|
||||
work_item=row["work_item"],
|
||||
actor=row["actor"],
|
||||
)
|
||||
|
||||
|
||||
class Transcript:
|
||||
"""One work item's log, as one actor reads and writes it (L6, L7).
|
||||
|
||||
Handed to each component per call by the orchestrator, which knows the
|
||||
active work item, so a component can neither read another item's events
|
||||
nor write without attribution.
|
||||
"""
|
||||
|
||||
def __init__(self, store: StateStore, work_item: str | None, actor: str) -> None:
|
||||
self._store = store
|
||||
self.work_item = work_item
|
||||
self.actor = actor
|
||||
|
||||
def append(self, event: CanonicalEvent) -> int:
|
||||
"""Committed and fsynced before return, like every durable write."""
|
||||
return self._store._append(event, self.work_item, self.actor)
|
||||
|
||||
def read(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""The log as recorded: every role's events, for the record."""
|
||||
return self._store._read(self.work_item, after)
|
||||
|
||||
def view(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""The log as this actor is shown it: what a model's context is
|
||||
built from (L6). See ``views``."""
|
||||
return project(self.read(after), self.actor, self._store.SHOWN)
|
||||
|
||||
def as_actor(self, actor: str) -> "Transcript":
|
||||
"""The same work item, written by another role: e.g. the executor
|
||||
handing a sub-task to the helper."""
|
||||
return self._store.transcript(self.work_item, actor)
|
||||
|
||||
def begin_intent(self, kind: str, payload: dict[str, Any], *, cause: str) -> int:
|
||||
"""An intent owned by this transcript's work item."""
|
||||
return self._store.begin_intent(
|
||||
kind, payload, cause=cause, work_item=self.work_item
|
||||
)
|
||||
@@ -14,7 +14,7 @@ from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_core.execution import SSHHost
|
||||
|
||||
SSHD = "/usr/sbin/sshd"
|
||||
|
||||
File renamed without changes.
@@ -62,8 +62,8 @@ class Vault:
|
||||
root: Path,
|
||||
folders: Mapping[str, str],
|
||||
*,
|
||||
agent: str, # whose writes these are, for provenance
|
||||
ignore: Sequence[str] = DEFAULT_IGNORE,
|
||||
agent: str = "nisse-dev",
|
||||
clock: Any = None,
|
||||
) -> None:
|
||||
self.root = Path(root)
|
||||
@@ -17,9 +17,10 @@ ranges still line up with the log. Everything not the viewer's own becomes a
|
||||
``UserMessage``: from where the viewer sits, it is all input.
|
||||
"""
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import replace
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
ActionVerdict,
|
||||
AssistantText,
|
||||
DelegationCompleted,
|
||||
@@ -36,18 +37,6 @@ from nisse_dev.canonical import (
|
||||
# How much of another role's tool result a narration line keeps.
|
||||
NARRATION_RESULT_CHARS = 300
|
||||
|
||||
# The roles whose work the viewer is shown, and how: "report" keeps only the
|
||||
# handback, "narrate" keeps a line per step. Anything not listed is not shown.
|
||||
_SHOWN: dict[str, dict[str, str]] = {
|
||||
"driver": {"helper": "report"},
|
||||
"helper": {"driver": "narrate"},
|
||||
# The toolchain step works in its own transcript: the driver sees only
|
||||
# its report (a tool result, or facts), the step the driver's work.
|
||||
"toolchain": {"driver": "narrate"},
|
||||
# The memory pass at an item's close (L6.4): the item as it went.
|
||||
"memory": {"driver": "narrate"},
|
||||
}
|
||||
|
||||
|
||||
def operator_label(source: str) -> str:
|
||||
"""Where an operator message came from, for its label."""
|
||||
@@ -90,11 +79,18 @@ def _narration(call: ToolCall, result: ToolResult | None) -> str:
|
||||
return f"{step} → {outcome}: {text}" if text else f"{step} → {outcome}"
|
||||
|
||||
|
||||
def project(events: list[StoredEvent], viewer: str) -> list[StoredEvent]:
|
||||
"""The events as ``viewer`` should be shown them, in log order."""
|
||||
def project(
|
||||
events: list[StoredEvent],
|
||||
viewer: str,
|
||||
shown_by_viewer: Mapping[str, Mapping[str, str]],
|
||||
) -> list[StoredEvent]:
|
||||
"""The events as ``viewer`` should be shown them, in log order.
|
||||
``shown_by_viewer``: for each viewer, the roles whose work it is shown,
|
||||
and how: "report" keeps only the handback, "narrate" keeps a line per
|
||||
step. Anything not listed is not shown."""
|
||||
# Verdicts are records about calls, never part of any role's context.
|
||||
events = [e for e in events if not isinstance(e.event, ActionVerdict)]
|
||||
shown = _SHOWN.get(viewer)
|
||||
shown = shown_by_viewer.get(viewer)
|
||||
if shown is None:
|
||||
# A role with no view defined (the expert is stateless, the
|
||||
# classifier sees only an action) reads the log as it is, with
|
||||
File renamed without changes.
@@ -3,7 +3,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
Compaction,
|
||||
DelegationCompleted,
|
||||
@@ -16,7 +16,7 @@ from nisse_dev.canonical import (
|
||||
from_payload,
|
||||
mint_call_id,
|
||||
)
|
||||
from nisse_dev.state import StateSchemaError, StateStore
|
||||
from nisse_core.state import StateSchemaError, StateStore
|
||||
|
||||
ALL_EVENTS = [
|
||||
UserMessage(text="use demo", source="matrix:$abc"),
|
||||
@@ -45,12 +45,12 @@ ALL_EVENTS = [
|
||||
def test_all_event_types_round_trip(tmp_path: Path) -> None:
|
||||
with StateStore(tmp_path / "state.db") as store:
|
||||
for event in ALL_EVENTS:
|
||||
store.transcript("t", "executor").append(event)
|
||||
store.transcript("t", "driver").append(event)
|
||||
with StateStore(tmp_path / "state.db") as store:
|
||||
stored = store.transcript("t", "executor").read()
|
||||
stored = store.transcript("t", "driver").read()
|
||||
assert [s.event for s in stored] == ALL_EVENTS
|
||||
assert [s.id for s in stored] == sorted(s.id for s in stored)
|
||||
assert {(s.work_item, s.actor) for s in stored} == {("t", "executor")}
|
||||
assert {(s.work_item, s.actor) for s in stored} == {("t", "driver")}
|
||||
|
||||
|
||||
def test_read_events_after_id(tmp_path: Path) -> None:
|
||||
@@ -8,7 +8,7 @@ route has to come from what LiteLLM says the model is.
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from nisse_dev.catalog import (
|
||||
from nisse_core.catalog import (
|
||||
CatalogUnavailable,
|
||||
ModelCatalog,
|
||||
UnknownModel,
|
||||
@@ -6,7 +6,7 @@ from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
StoredEvent,
|
||||
@@ -14,8 +14,8 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.catalog import ModelCatalog, UnknownLimits
|
||||
from nisse_dev.context import (
|
||||
from nisse_core.catalog import ModelCatalog, UnknownLimits
|
||||
from nisse_core.context import (
|
||||
Budget,
|
||||
BudgetedClient,
|
||||
ContextOverflow,
|
||||
@@ -23,9 +23,9 @@ from nisse_dev.context import (
|
||||
fit,
|
||||
head_and_tail,
|
||||
)
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import StateStore
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
|
||||
def stored(events: list[CanonicalEvent]) -> list[StoredEvent]:
|
||||
@@ -1,10 +1,10 @@
|
||||
import io
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_dev.canonical import Fact, PlanSubmitted, ToolCall, ToolResult, UserMessage
|
||||
from nisse_dev.dump import main, render
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_core.canonical import Fact, PlanSubmitted, ToolCall, ToolResult, UserMessage
|
||||
from nisse_core.dump import main, render
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import StateStore
|
||||
|
||||
|
||||
def store_with_a_run(path: Path) -> None:
|
||||
@@ -5,7 +5,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.execution import REDACTED, HostUnavailable, SSHHost, redact
|
||||
from nisse_core.execution import REDACTED, HostUnavailable, SSHHost, redact
|
||||
|
||||
|
||||
def test_exit_codes_and_streams_come_back(ssh_host: SSHHost) -> None:
|
||||
@@ -82,7 +82,7 @@ def test_a_changed_host_key_is_its_own_failure_and_only_forgetting_clears_it(
|
||||
) -> None:
|
||||
import subprocess
|
||||
|
||||
from nisse_dev.execution import HostKeyChanged
|
||||
from nisse_core.execution import HostKeyChanged
|
||||
|
||||
host = SSHHost(
|
||||
ssh_host.address,
|
||||
@@ -3,14 +3,14 @@ import json
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from nisse_dev.gitea import (
|
||||
from nisse_core.gitea import (
|
||||
PR_PAGE_SIZE,
|
||||
Comment,
|
||||
GiteaClient,
|
||||
GiteaError,
|
||||
pull_request_reconciler,
|
||||
)
|
||||
from nisse_dev.state import Intent
|
||||
from nisse_core.state import Intent
|
||||
|
||||
INTENT = Intent(
|
||||
id=1,
|
||||
@@ -0,0 +1,25 @@
|
||||
"""nisse-core is what every agent is built on (Part L): it never imports an
|
||||
agent."""
|
||||
|
||||
import ast
|
||||
from pathlib import Path
|
||||
|
||||
import nisse_core
|
||||
|
||||
AGENTS = ("nisse_dev", "nisse_ops")
|
||||
|
||||
|
||||
def test_core_imports_no_agent() -> None:
|
||||
root = Path(nisse_core.__file__).parent
|
||||
found = []
|
||||
for path in sorted(root.rglob("*.py")):
|
||||
for node in ast.walk(ast.parse(path.read_text())):
|
||||
if isinstance(node, ast.Import):
|
||||
names = [a.name for a in node.names]
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
names = [node.module or ""]
|
||||
else:
|
||||
continue
|
||||
found += [f"{path.name}: {n}" for n in names if n.split(".")[0] in AGENTS]
|
||||
assert len(list(root.rglob("*.py"))) > 10 # the modules really are here
|
||||
assert found == []
|
||||
@@ -5,14 +5,14 @@ from typing import Any
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
StoredEvent,
|
||||
ToolCall,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.envfile import load_env_file
|
||||
from nisse_dev.llm import (
|
||||
from nisse_core.envfile import load_env_file
|
||||
from nisse_core.llm import (
|
||||
DEFAULT_MAX_OUTPUT_TOKENS,
|
||||
OUTAGE_BACKOFF_S,
|
||||
RATE_LIMIT_RETRY_S,
|
||||
@@ -4,8 +4,8 @@ import threading
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from nisse_dev.matrix import MatrixConfig, MatrixTransport, TransportError
|
||||
from nisse_dev.transport import InboundMessage
|
||||
from nisse_core.matrix import MatrixConfig, MatrixTransport, TransportError
|
||||
from nisse_core.transport import InboundMessage
|
||||
|
||||
CONFIG = MatrixConfig(
|
||||
homeserver="https://matrix.test",
|
||||
@@ -349,7 +349,7 @@ def test_a_reply_to_the_operators_own_threaded_message_joins_the_thread() -> Non
|
||||
|
||||
|
||||
def test_in_memory_transport_plays_threads() -> None:
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
|
||||
transport = InMemoryTransport()
|
||||
seen: list[InboundMessage] = []
|
||||
@@ -5,15 +5,17 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.memory import Memory
|
||||
from nisse_dev.vault import Vault, VaultError, split_frontmatter
|
||||
from nisse_core.memory import Memory
|
||||
from nisse_core.vault import Vault, VaultError, split_frontmatter
|
||||
|
||||
FOLDER = "memory/nisse-dev"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def memory(tmp_path: Path) -> Memory:
|
||||
return Memory(Vault(tmp_path / "v", {FOLDER: "rw", "notes": "ro"}), FOLDER)
|
||||
return Memory(
|
||||
Vault(tmp_path / "v", {FOLDER: "rw", "notes": "ro"}, agent="nisse-dev"), FOLDER
|
||||
)
|
||||
|
||||
|
||||
def note(memory: Memory, name: str) -> tuple[dict, str]:
|
||||
@@ -8,9 +8,9 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.reconcile import reconcile_on_boot
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.reconcile import reconcile_on_boot
|
||||
from nisse_core.state import StateStore
|
||||
|
||||
ENVELOPE_FIELDS = {"ts", "agent", "event", "task_id", "cause"}
|
||||
|
||||
@@ -132,7 +132,7 @@ def test_literal_log_lines_for_human_readability(
|
||||
|
||||
return datetime(2026, 9, 5, 12, 0, 0, tzinfo=UTC)
|
||||
|
||||
monkeypatch.setattr("nisse_dev.oplog.datetime", _FrozenDatetime)
|
||||
monkeypatch.setattr("nisse_core.oplog.datetime", _FrozenDatetime)
|
||||
|
||||
start_item(store, "demo-m2", scope="demo", cause="issue assigned: demo#2")
|
||||
store.oplog.task_id = "demo-m2"
|
||||
@@ -2,7 +2,7 @@ import json
|
||||
|
||||
import httpx
|
||||
|
||||
from nisse_dev.probe import interpret, probe
|
||||
from nisse_core.probe import interpret, probe
|
||||
|
||||
|
||||
def test_a_cut_at_the_declared_window_matches() -> None:
|
||||
@@ -2,8 +2,8 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.reconcile import reconcile_on_boot
|
||||
from nisse_dev.state import Intent, StateStore
|
||||
from nisse_core.reconcile import reconcile_on_boot
|
||||
from nisse_core.state import Intent, StateStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -1,7 +1,7 @@
|
||||
import json
|
||||
from itertools import pairwise
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
Compaction,
|
||||
GateOpened,
|
||||
@@ -10,7 +10,7 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.render import OpenAIChatRenderer, effective_events
|
||||
from nisse_core.render import OpenAIChatRenderer, effective_events
|
||||
|
||||
|
||||
def stored(events: list) -> list[StoredEvent]:
|
||||
+3
-3
@@ -4,7 +4,7 @@ these tests pin the differences that a normalising middlebox would blur."""
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
Compaction,
|
||||
StoredEvent,
|
||||
@@ -12,8 +12,8 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.llm import AnthropicPassthroughClient, LLMError
|
||||
from nisse_dev.render_anthropic import (
|
||||
from nisse_core.llm import AnthropicPassthroughClient, LLMError
|
||||
from nisse_core.render_anthropic import (
|
||||
AnthropicMessagesRenderer,
|
||||
to_anthropic_tools,
|
||||
)
|
||||
@@ -3,7 +3,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.state import StateSchemaError, StateStore
|
||||
from nisse_core.state import StateSchemaError, StateStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -121,28 +121,3 @@ def test_newer_schema_is_refused(db: Path) -> None:
|
||||
pass
|
||||
with pytest.raises(StateSchemaError, match="newer"):
|
||||
StateStore(db, schema_version=1)
|
||||
|
||||
|
||||
def test_a_vm_block_persists_across_reopen(db: Path) -> None:
|
||||
with StateStore(db) as store:
|
||||
store.block_vm("root@vm1", cause="broken", reason="pip crashes", work_item="x")
|
||||
store.block_vm("root@vm1", cause="broken", reason="later", work_item="y")
|
||||
with StateStore(db) as store:
|
||||
[block] = store.vm_blocks().values()
|
||||
assert (block.host, block.reason, block.work_item) == (
|
||||
"root@vm1",
|
||||
"pip crashes", # the first block is kept
|
||||
"x",
|
||||
)
|
||||
assert store.unblock_vm("root@vm1", cause="test") == block
|
||||
assert store.unblock_vm("root@vm1", cause="test") is None
|
||||
assert store.vm_blocks() == {}
|
||||
|
||||
|
||||
def test_a_version_4_store_gains_vm_blocks(db: Path) -> None:
|
||||
with StateStore(db, schema_version=4) as store:
|
||||
store._conn.execute("DROP TABLE vm_blocks") # as a real version 4 store
|
||||
with StateStore(db) as store:
|
||||
assert store.schema_version == 5
|
||||
store.block_vm("root@vm1", cause="broken", reason="r", work_item=None)
|
||||
assert list(store.vm_blocks()) == ["root@vm1"]
|
||||
@@ -9,8 +9,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from nisse_dev import vault as vault_module
|
||||
from nisse_dev.vault import Vault, VaultError, split_frontmatter
|
||||
from nisse_core import vault as vault_module
|
||||
from nisse_core.vault import Vault, VaultError, split_frontmatter
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -32,6 +32,7 @@ def vault(tmp_path: Path) -> Vault:
|
||||
return Vault(
|
||||
root,
|
||||
{"memory/nisse-dev": "rw", "notes": "ro"},
|
||||
agent="nisse-dev",
|
||||
clock=lambda: datetime(2026, 10, 8, 12, 0, tzinfo=UTC),
|
||||
)
|
||||
|
||||
@@ -138,7 +139,7 @@ def test_bad_frontmatter_is_refused(vault: Vault) -> None:
|
||||
|
||||
|
||||
def test_read_write_folders_are_made(tmp_path: Path) -> None:
|
||||
Vault(tmp_path / "v", {"memory/nisse-dev": "rw", "notes": "ro"})
|
||||
Vault(tmp_path / "v", {"memory/nisse-dev": "rw", "notes": "ro"}, agent="a")
|
||||
assert (tmp_path / "v" / "memory" / "nisse-dev").is_dir()
|
||||
assert not (tmp_path / "v" / "notes").exists() # not ours to make
|
||||
assert yaml # frontmatter is YAML
|
||||
@@ -1,9 +1,10 @@
|
||||
"""Per-role transcript views (architecture L6; Development Plan M1.2)."""
|
||||
|
||||
import io
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
@@ -14,14 +15,24 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.render import OpenAIChatRenderer
|
||||
from nisse_dev.render_anthropic import AnthropicMessagesRenderer
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.render import OpenAIChatRenderer
|
||||
from nisse_core.render_anthropic import AnthropicMessagesRenderer
|
||||
from nisse_core.state import StateStore
|
||||
|
||||
|
||||
class Store(StateStore):
|
||||
"""An agent's store: the helper is shown the driver's work as narration,
|
||||
the driver only the helper's report."""
|
||||
|
||||
SHOWN: Mapping[str, Mapping[str, str]] = {
|
||||
"driver": {"helper": "report"},
|
||||
"helper": {"driver": "narrate"},
|
||||
}
|
||||
|
||||
|
||||
def new_store(tmp_path: Path) -> StateStore:
|
||||
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||
return Store(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||
|
||||
|
||||
def texts(events: list[StoredEvent]) -> list[str]:
|
||||
@@ -4,9 +4,9 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.worker import InlineWorker, SerialWorker
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import StateStore
|
||||
from nisse_core.worker import InlineWorker, SerialWorker
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -11,8 +11,7 @@ from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.bench.tasks import Task
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
CanonicalEvent,
|
||||
Fact,
|
||||
StoredEvent,
|
||||
@@ -20,9 +19,11 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.catalog import ModelFacts
|
||||
from nisse_core.catalog import ModelFacts
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_dev.bench.tasks import Task
|
||||
from nisse_dev.config import Tuning
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.loop.agent import SYSTEM
|
||||
from nisse_dev.loop.driver import Driver, RunOutcome
|
||||
from nisse_dev.loop.tools import (
|
||||
@@ -31,7 +32,6 @@ from nisse_dev.loop.tools import (
|
||||
FINISH_MILESTONE,
|
||||
START_MILESTONE,
|
||||
)
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import EXECUTION_TOOLS, Workspace
|
||||
|
||||
|
||||
@@ -23,9 +23,9 @@ from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from nisse_dev.llm import DEFAULT_MAX_OUTPUT_TOKENS, DEFAULT_REQUEST_TIMEOUT_S, RoleMap
|
||||
from nisse_dev.matrix import MatrixConfig
|
||||
from nisse_dev.vault import ACCESS, DEFAULT_IGNORE
|
||||
from nisse_core.llm import DEFAULT_MAX_OUTPUT_TOKENS, DEFAULT_REQUEST_TIMEOUT_S, RoleMap
|
||||
from nisse_core.matrix import MatrixConfig
|
||||
from nisse_core.vault import ACCESS, DEFAULT_IGNORE
|
||||
|
||||
DEFAULT_CONFIG_FILE = "nisse-dev.yaml"
|
||||
|
||||
|
||||
@@ -28,8 +28,8 @@ from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Literal
|
||||
|
||||
from nisse_dev.cancel import Cancel, skip_calls
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.cancel import Cancel, skip_calls
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
@@ -39,10 +39,11 @@ from nisse_dev.canonical import (
|
||||
ToolCall,
|
||||
ToolResult,
|
||||
)
|
||||
from nisse_core.llm import LLMClient, LLMError, UpstreamUnavailable
|
||||
from nisse_core.reconcile import Reconciler
|
||||
from nisse_core.state import Intent, Transcript
|
||||
from nisse_dev.git import GitError, GitRepo
|
||||
from nisse_dev.llm import LLMClient, LLMError, UpstreamUnavailable
|
||||
from nisse_dev.reconcile import Reconciler
|
||||
from nisse_dev.state import Intent, StateStore, Transcript
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import EXECUTION_TOOLS, MUTATING_TOOLS, Workspace
|
||||
|
||||
DELEGATION_INTENT = "delegation"
|
||||
|
||||
@@ -15,7 +15,7 @@ from collections.abc import Iterable
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
ActionVerdict,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
|
||||
@@ -26,14 +26,14 @@ import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
StoredEvent,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||
from nisse_dev.state import Transcript
|
||||
from nisse_core.llm import LLMClient, LLMError, RoleMap
|
||||
from nisse_core.state import Transcript
|
||||
|
||||
EXPERT_ROLE = "expert"
|
||||
DEFAULT_BUDGET = 3
|
||||
|
||||
@@ -21,7 +21,7 @@ import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_dev.execution import HostUnavailable, SSHHost
|
||||
from nisse_core.execution import HostUnavailable, SSHHost
|
||||
|
||||
TIMEOUT_S = 120
|
||||
|
||||
|
||||
@@ -34,8 +34,8 @@ from nisse_core.actions import (
|
||||
Action,
|
||||
Verdict,
|
||||
)
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.canonical import (
|
||||
ActionVerdict,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
@@ -46,13 +46,19 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_core.execution import HostKeyChanged, HostUnavailable
|
||||
from nisse_core.gitea import GiteaClient, GiteaError, Issue, PullRequest
|
||||
from nisse_core.llm import LLMError, RoleMap
|
||||
from nisse_core.memory import Memory
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import WorkItem
|
||||
from nisse_core.transport import InboundMessage, Transport
|
||||
from nisse_core.vault import Vault, VaultError
|
||||
from nisse_core.worker import Worker
|
||||
from nisse_dev.config import GENERAL
|
||||
from nisse_dev.delegation import DELEGATION_INTENT, Delegation
|
||||
from nisse_dev.execution import HostKeyChanged, HostUnavailable
|
||||
from nisse_dev.expert import BudgetExhausted, ExpertDesk
|
||||
from nisse_dev.git import GitError
|
||||
from nisse_dev.gitea import GiteaClient, GiteaError, Issue, PullRequest
|
||||
from nisse_dev.llm import LLMError, RoleMap
|
||||
from nisse_dev.loop import preflight
|
||||
from nisse_dev.loop.actions import classify, commits
|
||||
from nisse_dev.loop.classifier import IN_SCOPE, ActionClassifier, Scope
|
||||
@@ -99,13 +105,8 @@ from nisse_dev.loop.tools import (
|
||||
TOOLCHAIN_STEP,
|
||||
WAIT_FOR_REVIEW,
|
||||
)
|
||||
from nisse_dev.memory import Memory
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore, WorkItem
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import EXECUTION_TOOLS, Workspace
|
||||
from nisse_dev.transport import InboundMessage, Transport
|
||||
from nisse_dev.vault import Vault, VaultError
|
||||
from nisse_dev.worker import Worker
|
||||
|
||||
INTAKE_INTERVAL_S = 30.0
|
||||
_GO = frozenset({"go", "approve"})
|
||||
|
||||
@@ -7,29 +7,29 @@ import threading
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.catalog import ModelCatalog, select_client
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.catalog import ModelCatalog, select_client
|
||||
from nisse_core.context import BudgetedClient
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.gitea import GiteaClient
|
||||
from nisse_core.llm import AnthropicPassthroughClient, LiteLLMClient
|
||||
from nisse_core.matrix import MatrixTransport
|
||||
from nisse_core.memory import Memory
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.vault import Vault
|
||||
from nisse_core.worker import SerialWorker
|
||||
from nisse_dev.config import AgentConfig, ConfigError
|
||||
from nisse_dev.config import Project as ProjectEntry
|
||||
from nisse_dev.context import BudgetedClient
|
||||
from nisse_dev.delegation import Caps
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.git import GitRepo, HTTPSToken, PushCredential, SSHKey
|
||||
from nisse_dev.gitea import GiteaClient
|
||||
from nisse_dev.llm import AnthropicPassthroughClient, LiteLLMClient
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.classifier import ActionClassifier
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.help import Helpers
|
||||
from nisse_dev.loop.milestones import Milestones
|
||||
from nisse_dev.loop.reader import GoAheadReader
|
||||
from nisse_dev.matrix import MatrixTransport
|
||||
from nisse_dev.memory import Memory
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.vault import Vault
|
||||
from nisse_dev.worker import SerialWorker
|
||||
|
||||
# How long shutdown waits for running work before leaving the store to the
|
||||
# durability layer (L8). A milestone can run far longer than a service
|
||||
@@ -301,7 +301,12 @@ def build_loop_agent(
|
||||
)
|
||||
transport = MatrixTransport(config.matrix)
|
||||
vault = (
|
||||
Vault(config.vault.root, config.vault.folders, ignore=config.vault.ignore)
|
||||
Vault(
|
||||
config.vault.root,
|
||||
config.vault.folders,
|
||||
agent="nisse-dev",
|
||||
ignore=config.vault.ignore,
|
||||
)
|
||||
if config.vault
|
||||
else None
|
||||
)
|
||||
|
||||
@@ -17,8 +17,8 @@ from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from nisse_core.actions import CONTAINED, DANGEROUS, Action, Verdict
|
||||
from nisse_dev.canonical import StoredEvent, ToolCall, UserMessage
|
||||
from nisse_dev.llm import LLMClient, LLMError
|
||||
from nisse_core.canonical import StoredEvent, ToolCall, UserMessage
|
||||
from nisse_core.llm import LLMClient, LLMError
|
||||
|
||||
SAFE = "safe"
|
||||
IN_SCOPE = "in_scope"
|
||||
|
||||
@@ -11,11 +11,11 @@ from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Literal
|
||||
|
||||
from nisse_dev.cancel import Cancel, skip_calls
|
||||
from nisse_dev.canonical import AssistantText, Fact, ToolCall, ToolResult
|
||||
from nisse_dev.llm import LLMClient, LLMError
|
||||
from nisse_core.cancel import Cancel, skip_calls
|
||||
from nisse_core.canonical import AssistantText, Fact, ToolCall, ToolResult
|
||||
from nisse_core.llm import LLMClient, LLMError
|
||||
from nisse_core.state import Transcript
|
||||
from nisse_dev.loop.tools import WAITING, names
|
||||
from nisse_dev.state import Transcript
|
||||
from nisse_dev.tools import Workspace
|
||||
|
||||
DEFAULT_MAX_TURNS = 30
|
||||
|
||||
@@ -11,10 +11,10 @@ half-edited tree).
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from nisse_core.llm import LLMClient, RoleMap
|
||||
from nisse_dev.config import Profile
|
||||
from nisse_dev.delegation import Caps
|
||||
from nisse_dev.expert import DEFAULT_BUDGET
|
||||
from nisse_dev.llm import LLMClient, RoleMap
|
||||
from nisse_dev.loop.tools import _tool
|
||||
|
||||
DELEGATE = _tool(
|
||||
|
||||
@@ -8,7 +8,8 @@ agreement, what it is waiting for) is in ``data["loop"]``.
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.state import StateStore, WorkItem
|
||||
from nisse_core.state import WorkItem
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
READY = "ready"
|
||||
RUNNING = "running" # a driver run is under way; on boot it means interrupted
|
||||
|
||||
@@ -13,8 +13,8 @@ import uuid
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from nisse_core.gitea import GiteaClient, GiteaError, PullRequest
|
||||
from nisse_dev.git import GitError, GitRepo, PushCredential
|
||||
from nisse_dev.gitea import GiteaClient, GiteaError, PullRequest
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
OPEN_PR = "open_pr"
|
||||
|
||||
@@ -21,7 +21,7 @@ from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from nisse_core.actions import split_shell, unwrap
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_core.execution import SSHHost
|
||||
|
||||
# What the harness itself runs on a build VM (L13, D4): part of the
|
||||
# template, never of a project's toolchain.
|
||||
|
||||
@@ -10,8 +10,8 @@ The driver's model reads for now; the classifier role takes over in M2.6.
|
||||
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.canonical import StoredEvent, ToolCall, UserMessage
|
||||
from nisse_dev.llm import LLMClient, LLMError
|
||||
from nisse_core.canonical import StoredEvent, ToolCall, UserMessage
|
||||
from nisse_core.llm import LLMClient, LLMError
|
||||
|
||||
|
||||
def command_form(text: str) -> str:
|
||||
|
||||
@@ -12,8 +12,8 @@ import sys
|
||||
from pathlib import Path
|
||||
from types import FrameType
|
||||
|
||||
from nisse_core.envfile import load_env_file
|
||||
from nisse_dev.config import AgentConfig, ConfigError
|
||||
from nisse_dev.envfile import load_env_file
|
||||
from nisse_dev.loop.build import build_loop_agent
|
||||
|
||||
|
||||
|
||||
@@ -1,42 +1,12 @@
|
||||
"""Durable state store (architecture L8).
|
||||
"""nisse-dev's state store (architecture L8): the shared one, with the dev
|
||||
agent's own roles, views and tables (build VM blocks, D3)."""
|
||||
|
||||
SQLite in WAL mode with ``synchronous=FULL``: every commit is fsynced, so a
|
||||
power cut mid-write can never produce torn state. Every state transition and
|
||||
intent-log write is a single immediately-committed statement — in-memory state
|
||||
that hasn't hit disk is assumed lost.
|
||||
|
||||
The intent log records a unit of work *before* it starts, so a restart knows
|
||||
what was in flight (see ``nisse_dev.reconcile`` for how pending intents are
|
||||
resolved against external truth on boot).
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from collections.abc import Callable, Mapping
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Self
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
CanonicalEvent,
|
||||
StoredEvent,
|
||||
UserMessage,
|
||||
event_type,
|
||||
from_payload,
|
||||
to_payload,
|
||||
)
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.views import project
|
||||
|
||||
CURRENT_SCHEMA_VERSION = 5
|
||||
|
||||
# Stores older than this predate work items (Development Plan M1.1). There was
|
||||
# no production data to carry over, so there is no migration from them: start
|
||||
# a fresh store (scripts/bringup.sh start sets the old one aside).
|
||||
OLDEST_SUPPORTED_VERSION = 4
|
||||
|
||||
Migration = Callable[[sqlite3.Connection], None]
|
||||
from nisse_core import state
|
||||
from nisse_core.state import Migration, utc_now
|
||||
|
||||
_VM_BLOCKS = """
|
||||
CREATE TABLE vm_blocks (
|
||||
@@ -53,46 +23,6 @@ def _add_vm_blocks(conn: sqlite3.Connection) -> None:
|
||||
conn.execute(_VM_BLOCKS)
|
||||
|
||||
|
||||
# Maps version N to the migration that takes the schema from N to N+1.
|
||||
MIGRATIONS: Mapping[int, Migration] = {4: _add_vm_blocks}
|
||||
|
||||
# Who produced an event (L7): the operator, the orchestrator's own code, or a
|
||||
# model role (L9). planner and executor are the old loop's (until M2.8), kept
|
||||
# so its records still read.
|
||||
ACTORS = frozenset(
|
||||
{
|
||||
"operator",
|
||||
"orchestrator",
|
||||
"planner",
|
||||
"executor",
|
||||
"driver",
|
||||
"helper",
|
||||
"expert",
|
||||
"classifier",
|
||||
"toolchain", # the toolchain step (D3): the driver's model, apart
|
||||
"memory", # the memory pass at an item's close (L6.4), apart too
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class StateSchemaError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkItem:
|
||||
id: str
|
||||
scope: str | None
|
||||
title: str
|
||||
status: str # open | closed
|
||||
outcome: str | None # completed | abandoned, once closed
|
||||
position: str # where the loop is for this item
|
||||
data: dict[str, Any]
|
||||
opened_at: str
|
||||
updated_at: str
|
||||
closed_at: str | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VMBlock:
|
||||
"""A build VM nothing may run on until it is recovered (D3): every
|
||||
@@ -105,298 +35,30 @@ class VMBlock:
|
||||
since: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Intent:
|
||||
id: int
|
||||
kind: str
|
||||
payload: dict[str, Any]
|
||||
status: str
|
||||
created_at: str
|
||||
resolved_at: str | None
|
||||
work_item: str | None = None
|
||||
|
||||
|
||||
_SCHEMA = [
|
||||
"CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)",
|
||||
"""
|
||||
CREATE TABLE work_items (
|
||||
id TEXT PRIMARY KEY,
|
||||
scope TEXT,
|
||||
title TEXT NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('open', 'closed')),
|
||||
outcome TEXT,
|
||||
position TEXT NOT NULL,
|
||||
data TEXT NOT NULL DEFAULT '{}',
|
||||
opened_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
closed_at TEXT
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE agent (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
active_item TEXT REFERENCES work_items(id),
|
||||
idle_from_event INTEGER NOT NULL DEFAULT 0,
|
||||
updated_at TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE intent_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
kind TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
created_at TEXT NOT NULL,
|
||||
resolved_at TEXT,
|
||||
work_item TEXT
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE canonical_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ts TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
work_item TEXT,
|
||||
actor TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
"CREATE INDEX canonical_events_by_item ON canonical_events (work_item, id)",
|
||||
"""
|
||||
CREATE TABLE eval_outcomes (
|
||||
event_id INTEGER PRIMARY KEY,
|
||||
verdict TEXT NOT NULL,
|
||||
note TEXT NOT NULL DEFAULT '',
|
||||
marked_at TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
_VM_BLOCKS,
|
||||
]
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(UTC).isoformat()
|
||||
|
||||
|
||||
class StateStore:
|
||||
def __init__(
|
||||
self,
|
||||
path: str | Path,
|
||||
*,
|
||||
schema_version: int = CURRENT_SCHEMA_VERSION,
|
||||
migrations: Mapping[int, Migration] = MIGRATIONS,
|
||||
oplog: OpLog | None = None,
|
||||
) -> None:
|
||||
# Log emission is owned by the store (L12): the same code path that
|
||||
# performs a durable write emits its log line, right after commit.
|
||||
self.oplog = oplog if oplog is not None else OpLog("nisse-dev")
|
||||
self._conn = sqlite3.connect(path, isolation_level=None)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA synchronous=FULL")
|
||||
self._conn.execute("PRAGMA busy_timeout=5000")
|
||||
self._target_version = schema_version
|
||||
self.closed = False
|
||||
self._init_or_migrate(migrations)
|
||||
|
||||
def _init_or_migrate(self, migrations: Mapping[int, Migration]) -> None:
|
||||
fresh = (
|
||||
self._conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name='meta'"
|
||||
).fetchone()
|
||||
is None
|
||||
)
|
||||
if fresh:
|
||||
self._in_transaction(self._create_schema)
|
||||
return
|
||||
stored = self.schema_version
|
||||
if stored < OLDEST_SUPPORTED_VERSION <= self._target_version:
|
||||
raise StateSchemaError(
|
||||
f"state store schema version {stored} predates work items "
|
||||
f"(version {OLDEST_SUPPORTED_VERSION}); start a fresh store "
|
||||
"(scripts/bringup.sh start sets the old one aside)"
|
||||
)
|
||||
if stored > self._target_version:
|
||||
raise StateSchemaError(
|
||||
f"state schema version {stored} is newer than this code "
|
||||
f"supports ({self._target_version}); refusing to open"
|
||||
)
|
||||
for version in range(stored, self._target_version):
|
||||
migrate = migrations.get(version)
|
||||
if migrate is None:
|
||||
raise StateSchemaError(
|
||||
f"no migration registered from schema version {version}"
|
||||
)
|
||||
self._in_transaction(lambda m=migrate, v=version: self._apply(m, v))
|
||||
self.oplog.emit(
|
||||
"schema.migrate",
|
||||
cause="startup",
|
||||
from_version=version,
|
||||
to_version=version + 1,
|
||||
)
|
||||
|
||||
def _create_schema(self) -> None:
|
||||
for statement in _SCHEMA:
|
||||
self._conn.execute(statement)
|
||||
self._conn.execute(
|
||||
"INSERT INTO meta (key, value) VALUES ('schema_version', ?)",
|
||||
(str(self._target_version),),
|
||||
)
|
||||
|
||||
def _apply(self, migrate: Migration, from_version: int) -> None:
|
||||
migrate(self._conn)
|
||||
self._conn.execute(
|
||||
"UPDATE meta SET value = ? WHERE key = 'schema_version'",
|
||||
(str(from_version + 1),),
|
||||
)
|
||||
|
||||
def _in_transaction(self, work: Callable[[], None]) -> None:
|
||||
self._conn.execute("BEGIN IMMEDIATE")
|
||||
try:
|
||||
work()
|
||||
except BaseException:
|
||||
self._conn.execute("ROLLBACK")
|
||||
raise
|
||||
self._conn.execute("COMMIT")
|
||||
|
||||
@property
|
||||
def schema_version(self) -> int:
|
||||
row = self._conn.execute(
|
||||
"SELECT value FROM meta WHERE key = 'schema_version'"
|
||||
).fetchone()
|
||||
return int(row["value"])
|
||||
|
||||
# --- work items (L1) ---------------------------------------------------
|
||||
|
||||
def _ensure_agent_row(self) -> None:
|
||||
self._conn.execute(
|
||||
"INSERT INTO agent (id, active_item, updated_at) VALUES (1, NULL, ?) "
|
||||
"ON CONFLICT(id) DO NOTHING",
|
||||
(_now(),),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _item(row: sqlite3.Row) -> WorkItem:
|
||||
return WorkItem(
|
||||
id=row["id"],
|
||||
scope=row["scope"],
|
||||
title=row["title"],
|
||||
status=row["status"],
|
||||
outcome=row["outcome"],
|
||||
position=row["position"],
|
||||
data=json.loads(row["data"]),
|
||||
opened_at=row["opened_at"],
|
||||
updated_at=row["updated_at"],
|
||||
closed_at=row["closed_at"],
|
||||
)
|
||||
|
||||
def work_item(self, item_id: str) -> WorkItem | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM work_items WHERE id = ?", (item_id,)
|
||||
).fetchone()
|
||||
return None if row is None else self._item(row)
|
||||
|
||||
def work_items(self, status: str | None = None) -> list[WorkItem]:
|
||||
query = "SELECT * FROM work_items"
|
||||
args: tuple[Any, ...] = ()
|
||||
if status is not None:
|
||||
query += " WHERE status = ?"
|
||||
args = (status,)
|
||||
rows = self._conn.execute(query + " ORDER BY opened_at", args).fetchall()
|
||||
return [self._item(row) for row in rows]
|
||||
|
||||
def active_item(self) -> WorkItem | None:
|
||||
row = self._conn.execute(
|
||||
"SELECT w.* FROM agent a JOIN work_items w ON w.id = a.active_item "
|
||||
"WHERE a.id = 1"
|
||||
).fetchone()
|
||||
return None if row is None else self._item(row)
|
||||
|
||||
def open_work_item(
|
||||
self,
|
||||
item_id: str,
|
||||
*,
|
||||
title: str,
|
||||
scope: str | None = None,
|
||||
position: str,
|
||||
data: dict[str, Any] | None = None,
|
||||
cause: str,
|
||||
) -> WorkItem:
|
||||
now = _now()
|
||||
self._conn.execute(
|
||||
"INSERT INTO work_items (id, scope, title, status, position, data, "
|
||||
"opened_at, updated_at) VALUES (?, ?, ?, 'open', ?, ?, ?, ?)",
|
||||
(item_id, scope, title, position, json.dumps(data or {}), now, now),
|
||||
)
|
||||
self.oplog.emit("work_item.open", cause=cause, work_item=item_id, scope=scope)
|
||||
item = self.work_item(item_id)
|
||||
assert item is not None
|
||||
return item
|
||||
|
||||
def set_active(self, item_id: str | None, *, cause: str) -> None:
|
||||
"""Point the agent at a work item, or at none (idle). Going idle
|
||||
records where in the log idleness began, so messages sent while idle
|
||||
can be carried into the next work item (carry_idle_messages)."""
|
||||
self._ensure_agent_row()
|
||||
previous = self._conn.execute(
|
||||
"SELECT active_item FROM agent WHERE id = 1"
|
||||
).fetchone()["active_item"]
|
||||
if item_id is None:
|
||||
last = self._conn.execute(
|
||||
"SELECT COALESCE(MAX(id), 0) AS last FROM canonical_events"
|
||||
).fetchone()["last"]
|
||||
self._conn.execute(
|
||||
"UPDATE agent SET active_item = NULL, idle_from_event = ?, "
|
||||
"updated_at = ? WHERE id = 1",
|
||||
(last, _now()),
|
||||
)
|
||||
else:
|
||||
self._conn.execute(
|
||||
"UPDATE agent SET active_item = ?, updated_at = ? WHERE id = 1",
|
||||
(item_id, _now()),
|
||||
)
|
||||
if previous != item_id:
|
||||
self.oplog.emit(
|
||||
"work_item.switch",
|
||||
cause=cause,
|
||||
**{"from": previous, "to": item_id},
|
||||
)
|
||||
|
||||
def update_work_item(
|
||||
self,
|
||||
item_id: str,
|
||||
*,
|
||||
position: str,
|
||||
data: dict[str, Any],
|
||||
cause: str,
|
||||
) -> None:
|
||||
"""Record a work item's new position and data; the full data is
|
||||
passed every time (no partial updates)."""
|
||||
previous = self.work_item(item_id)
|
||||
if previous is None:
|
||||
raise KeyError(f"no work item {item_id!r}")
|
||||
self._conn.execute(
|
||||
"UPDATE work_items SET position = ?, data = ?, updated_at = ? WHERE id = ?",
|
||||
(position, json.dumps(data), _now(), item_id),
|
||||
)
|
||||
self.oplog.emit(
|
||||
"state.transition",
|
||||
cause=cause,
|
||||
**{"from": previous.position, "to": position, "project": previous.scope},
|
||||
)
|
||||
|
||||
def close_work_item(self, item_id: str, *, outcome: str, cause: str) -> None:
|
||||
now = _now()
|
||||
row = self._conn.execute(
|
||||
"UPDATE work_items SET status = 'closed', outcome = ?, closed_at = ?, "
|
||||
"updated_at = ? WHERE id = ? AND status = 'open' RETURNING id",
|
||||
(outcome, now, now, item_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(f"no open work item {item_id!r}")
|
||||
self.oplog.emit(
|
||||
"work_item.close", cause=cause, work_item=item_id, outcome=outcome
|
||||
)
|
||||
class StateStore(state.StateStore):
|
||||
AGENT = "nisse-dev"
|
||||
SCHEMA_VERSION = 5
|
||||
SCHEMA = (_VM_BLOCKS,)
|
||||
MIGRATIONS: Mapping[int, Migration] = {4: _add_vm_blocks}
|
||||
# planner and executor are the old loop's (until M2.8), kept so its
|
||||
# records still read.
|
||||
ACTORS = state.ACTORS | {
|
||||
"planner",
|
||||
"executor",
|
||||
"toolchain", # the toolchain step (D3): the driver's model, apart
|
||||
"memory", # the memory pass at an item's close (L6.4), apart too
|
||||
}
|
||||
# What each role is shown of the others' work (``nisse_core.views``).
|
||||
SHOWN: Mapping[str, Mapping[str, str]] = {
|
||||
"driver": {"helper": "report"},
|
||||
"helper": {"driver": "narrate"},
|
||||
# The toolchain step works in its own transcript: the driver sees
|
||||
# only its report (a tool result, or facts), the step the driver's
|
||||
# work.
|
||||
"toolchain": {"driver": "narrate"},
|
||||
# The memory pass at an item's close (L6.4): the item as it went.
|
||||
"memory": {"driver": "narrate"},
|
||||
}
|
||||
|
||||
# --- build VMs (D3) ----------------------------------------------------
|
||||
|
||||
@@ -407,7 +69,7 @@ class StateStore:
|
||||
row = self._conn.execute(
|
||||
"INSERT INTO vm_blocks (host, cause, reason, work_item, since) "
|
||||
"VALUES (?, ?, ?, ?, ?) ON CONFLICT(host) DO NOTHING RETURNING host",
|
||||
(host, cause, reason, work_item, _now()),
|
||||
(host, cause, reason, work_item, utc_now()),
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
self.oplog.emit("vm.block", cause=cause, host=host, work_item=work_item)
|
||||
@@ -434,206 +96,3 @@ class StateStore:
|
||||
def vm_blocks(self) -> dict[str, VMBlock]:
|
||||
rows = self._conn.execute("SELECT * FROM vm_blocks").fetchall()
|
||||
return {row["host"]: VMBlock(**dict(row)) for row in rows}
|
||||
|
||||
def carry_idle_messages(self, item_id: str) -> int:
|
||||
"""Copy what the operator said while the agent was idle into a work
|
||||
item that has just opened (D1: messages sent before `use` are kept
|
||||
for it). Copied as new events, with the original's ID in the source,
|
||||
never by reassigning the originals: the log is never edited (L7)."""
|
||||
self._ensure_agent_row()
|
||||
since = self._conn.execute(
|
||||
"SELECT idle_from_event FROM agent WHERE id = 1"
|
||||
).fetchone()["idle_from_event"]
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item IS NULL "
|
||||
"AND actor = 'operator' AND type = 'user_message' AND id > ? ORDER BY id",
|
||||
(since,),
|
||||
).fetchall()
|
||||
log = self.transcript(item_id, "operator")
|
||||
for row in rows:
|
||||
payload = json.loads(row["payload"])
|
||||
log.append(
|
||||
UserMessage(
|
||||
text=payload["text"],
|
||||
source=f"{payload.get('source', '')} (sent while idle, #{row['id']})",
|
||||
)
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
# --- intents (L8) ------------------------------------------------------
|
||||
|
||||
def begin_intent(
|
||||
self,
|
||||
kind: str,
|
||||
payload: dict[str, Any],
|
||||
*,
|
||||
cause: str,
|
||||
work_item: str | None = None,
|
||||
) -> int:
|
||||
"""Durably record that a unit of work is about to start, owned by
|
||||
the work item it belongs to (L1)."""
|
||||
cursor = self._conn.execute(
|
||||
"INSERT INTO intent_log (kind, payload, created_at, work_item) "
|
||||
"VALUES (?, ?, ?, ?)",
|
||||
(kind, json.dumps(payload), _now(), work_item),
|
||||
)
|
||||
assert cursor.lastrowid is not None
|
||||
self.oplog.emit(
|
||||
"intent.begin", cause=cause, intent_id=cursor.lastrowid, kind=kind
|
||||
)
|
||||
return cursor.lastrowid
|
||||
|
||||
def commit_intent(self, intent_id: int, *, cause: str) -> None:
|
||||
self._resolve_intent(intent_id, "committed", "intent.commit", cause)
|
||||
|
||||
def abort_intent(self, intent_id: int, *, cause: str) -> None:
|
||||
self._resolve_intent(intent_id, "aborted", "intent.abort", cause)
|
||||
|
||||
def _resolve_intent(
|
||||
self, intent_id: int, status: str, event: str, cause: str
|
||||
) -> None:
|
||||
row = self._conn.execute(
|
||||
"UPDATE intent_log SET status = ?, resolved_at = ? "
|
||||
"WHERE id = ? AND status = 'pending' RETURNING kind",
|
||||
(status, _now(), intent_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(f"no pending intent with id {intent_id}")
|
||||
self.oplog.emit(event, cause=cause, intent_id=intent_id, kind=row["kind"])
|
||||
|
||||
def pending_intents(self) -> list[Intent]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM intent_log WHERE status = 'pending' ORDER BY id"
|
||||
).fetchall()
|
||||
return [
|
||||
Intent(
|
||||
id=row["id"],
|
||||
kind=row["kind"],
|
||||
payload=json.loads(row["payload"]),
|
||||
status=row["status"],
|
||||
created_at=row["created_at"],
|
||||
resolved_at=row["resolved_at"],
|
||||
work_item=row["work_item"],
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
||||
# --- the canonical log (L7) -------------------------------------------
|
||||
|
||||
def transcript(self, work_item: str | None, actor: str) -> "Transcript":
|
||||
"""The log as one role sees it for one work item: what it reads is
|
||||
that item's events only, and what it writes carries the item and the
|
||||
actor. None is the agent-level scope (L6)."""
|
||||
if actor not in ACTORS:
|
||||
raise ValueError(f"unknown actor {actor!r}")
|
||||
return Transcript(self, work_item, actor)
|
||||
|
||||
def all_events(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""Every event across work items, for the few readers whose job
|
||||
spans them (evaluation capture, dumps). Model calls never use this."""
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE id > ? ORDER BY id", (after,)
|
||||
).fetchall()
|
||||
return [_stored(row) for row in rows]
|
||||
|
||||
def _append(self, event: CanonicalEvent, work_item: str | None, actor: str) -> int:
|
||||
cursor = self._conn.execute(
|
||||
"INSERT INTO canonical_events (ts, type, payload, work_item, actor) "
|
||||
"VALUES (?, ?, ?, ?, ?)",
|
||||
(
|
||||
_now(),
|
||||
event_type(event),
|
||||
json.dumps(to_payload(event)),
|
||||
work_item,
|
||||
actor,
|
||||
),
|
||||
)
|
||||
assert cursor.lastrowid is not None
|
||||
return cursor.lastrowid
|
||||
|
||||
def _read(self, work_item: str | None, after: int) -> list[StoredEvent]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM canonical_events WHERE work_item IS ? AND id > ? "
|
||||
"ORDER BY id",
|
||||
(work_item, after),
|
||||
).fetchall()
|
||||
return [_stored(row) for row in rows]
|
||||
|
||||
def mark_outcome(self, event_id: int, verdict: str, note: str = "") -> None:
|
||||
"""Record whether a delegation or consult was good (L9).
|
||||
|
||||
Kept out of the canonical log on purpose: the log is an append-only
|
||||
record of what happened, and a verdict is a later judgement about it
|
||||
that may be revised.
|
||||
"""
|
||||
self._conn.execute(
|
||||
"INSERT INTO eval_outcomes (event_id, verdict, note, marked_at) "
|
||||
"VALUES (?, ?, ?, ?) ON CONFLICT(event_id) DO UPDATE SET "
|
||||
"verdict = excluded.verdict, note = excluded.note, "
|
||||
"marked_at = excluded.marked_at",
|
||||
(event_id, verdict, note, _now()),
|
||||
)
|
||||
|
||||
def outcomes(self) -> dict[int, tuple[str, str]]:
|
||||
rows = self._conn.execute(
|
||||
"SELECT event_id, verdict, note FROM eval_outcomes"
|
||||
).fetchall()
|
||||
return {r["event_id"]: (r["verdict"], r["note"]) for r in rows}
|
||||
|
||||
def close(self) -> None:
|
||||
self._conn.close()
|
||||
self.closed = True
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: object) -> None:
|
||||
self.close()
|
||||
|
||||
|
||||
def _stored(row: sqlite3.Row) -> StoredEvent:
|
||||
return StoredEvent(
|
||||
id=row["id"],
|
||||
ts=row["ts"],
|
||||
event=from_payload(row["type"], json.loads(row["payload"])),
|
||||
work_item=row["work_item"],
|
||||
actor=row["actor"],
|
||||
)
|
||||
|
||||
|
||||
class Transcript:
|
||||
"""One work item's log, as one actor reads and writes it (L6, L7).
|
||||
|
||||
Handed to each component per call by the orchestrator, which knows the
|
||||
active work item, so a component can neither read another item's events
|
||||
nor write without attribution.
|
||||
"""
|
||||
|
||||
def __init__(self, store: StateStore, work_item: str | None, actor: str) -> None:
|
||||
self._store = store
|
||||
self.work_item = work_item
|
||||
self.actor = actor
|
||||
|
||||
def append(self, event: CanonicalEvent) -> int:
|
||||
"""Committed and fsynced before return, like every durable write."""
|
||||
return self._store._append(event, self.work_item, self.actor)
|
||||
|
||||
def read(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""The log as recorded: every role's events, for the record."""
|
||||
return self._store._read(self.work_item, after)
|
||||
|
||||
def view(self, after: int = 0) -> list[StoredEvent]:
|
||||
"""The log as this actor is shown it: what a model's context is
|
||||
built from (L6). See ``views``."""
|
||||
return project(self.read(after), self.actor)
|
||||
|
||||
def as_actor(self, actor: str) -> "Transcript":
|
||||
"""The same work item, written by another role: e.g. the executor
|
||||
handing a sub-task to the helper."""
|
||||
return self._store.transcript(self.work_item, actor)
|
||||
|
||||
def begin_intent(self, kind: str, payload: dict[str, Any], *, cause: str) -> int:
|
||||
"""An intent owned by this transcript's work item."""
|
||||
return self._store.begin_intent(
|
||||
kind, payload, cause=cause, work_item=self.work_item
|
||||
)
|
||||
@@ -20,8 +20,8 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.context import head_and_tail
|
||||
from nisse_dev.execution import HostUnavailable, Result, SSHHost
|
||||
from nisse_core.context import head_and_tail
|
||||
from nisse_core.execution import HostUnavailable, Result, SSHHost
|
||||
|
||||
# Tools that change or verify something. Reading a file or listing a
|
||||
# directory is a model looking around, not doing the work, so neither counts
|
||||
|
||||
@@ -10,8 +10,8 @@ unit tests and be rejected live; this makes it fail here.
|
||||
|
||||
from typing import Any
|
||||
|
||||
from nisse_dev.render import OpenAIChatRenderer
|
||||
from nisse_dev.render_anthropic import AnthropicMessagesRenderer
|
||||
from nisse_core.render import OpenAIChatRenderer
|
||||
from nisse_core.render_anthropic import AnthropicMessagesRenderer
|
||||
|
||||
|
||||
def assert_renderable(events: list[Any]) -> None:
|
||||
|
||||
@@ -11,13 +11,13 @@ import httpx
|
||||
import pytest
|
||||
from test_config import SECRETS, P, tree
|
||||
|
||||
from nisse_dev.catalog import ModelCatalog, UnknownModel
|
||||
from nisse_core.catalog import ModelCatalog, UnknownModel
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.llm import AnthropicPassthroughClient, LiteLLMClient
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_dev.config import AgentConfig
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.llm import AnthropicPassthroughClient, LiteLLMClient
|
||||
from nisse_dev.loop.build import RunningLoop, build_loop_agent, discover_models
|
||||
from nisse_dev.loop.build import prepare_workspace as prepare
|
||||
from nisse_dev.oplog import OpLog
|
||||
|
||||
# What the test roles resolve to, as LiteLLM would report them.
|
||||
TEST_CATALOG = ModelCatalog.from_payload(
|
||||
@@ -143,7 +143,7 @@ def offline(monkeypatch: Any, origins: dict[str, str] | str, catalog=None) -> No
|
||||
"""Only the network boundaries faked: Gitea, LiteLLM's model list, Matrix."""
|
||||
# The proxy here doesn't exist: a call to it fails at once rather than
|
||||
# waiting out an outage's backoff (L5).
|
||||
monkeypatch.setattr("nisse_dev.llm.OUTAGE_BACKOFF_S", ())
|
||||
monkeypatch.setattr("nisse_core.llm.OUTAGE_BACKOFF_S", ())
|
||||
monkeypatch.setattr(
|
||||
"nisse_dev.loop.build.GiteaClient.whoami", lambda self: "nisse-dev-testing"
|
||||
)
|
||||
|
||||
@@ -7,6 +7,9 @@ from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_core.canonical import AssistantText, ToolCall
|
||||
from nisse_core.catalog import ModelFacts
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_dev.bench.results import (
|
||||
append,
|
||||
load,
|
||||
@@ -17,10 +20,7 @@ from nisse_dev.bench.results import (
|
||||
)
|
||||
from nisse_dev.bench.runner import MeteringClient, run_task
|
||||
from nisse_dev.bench.tasks import CHECK_DIR, load_tasks
|
||||
from nisse_dev.canonical import AssistantText, ToolCall
|
||||
from nisse_dev.catalog import ModelFacts
|
||||
from nisse_dev.config import DriverTuning, Tuning
|
||||
from nisse_dev.execution import SSHHost
|
||||
|
||||
# The build VM for every test here: the session's throwaway sshd.
|
||||
HOST: SSHHost
|
||||
|
||||
@@ -5,8 +5,8 @@ from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_core.llm import DEFAULT_REQUEST_TIMEOUT_S
|
||||
from nisse_dev.config import AgentConfig, ConfigError, Project
|
||||
from nisse_dev.llm import DEFAULT_REQUEST_TIMEOUT_S
|
||||
|
||||
# A complete config as the YAML file would give it, with secrets by reference.
|
||||
COMPLETE: dict[str, Any] = {
|
||||
@@ -142,7 +142,7 @@ def test_request_timeout_is_configurable_and_checked() -> None:
|
||||
|
||||
|
||||
def test_the_request_timeout_reaches_both_clients() -> None:
|
||||
from nisse_dev.catalog import ModelCatalog
|
||||
from nisse_core.catalog import ModelCatalog
|
||||
from nisse_dev.loop.build import make_llm_clients
|
||||
|
||||
normalized, passthrough = make_llm_clients(
|
||||
|
||||
@@ -4,8 +4,8 @@ from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.cancel import STOPPED_RESULT, Cancel
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.cancel import STOPPED_RESULT, Cancel
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
DelegationCompleted,
|
||||
@@ -14,6 +14,10 @@ from nisse_dev.canonical import (
|
||||
ToolCall,
|
||||
ToolResult,
|
||||
)
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.llm import BudgetExhausted, LLMError, UpstreamUnavailable
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import Transcript
|
||||
from nisse_dev.delegation import (
|
||||
DELEGATE_TOOLS,
|
||||
DELEGATION_INTENT,
|
||||
@@ -22,11 +26,8 @@ from nisse_dev.delegation import (
|
||||
DelegationResult,
|
||||
delegation_reconciler,
|
||||
)
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.git import GitRepo
|
||||
from nisse_dev.llm import BudgetExhausted, LLMError, UpstreamUnavailable
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore, Transcript
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import Workspace
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
"""nisse-dev's state store: the shared one, with its own roles, views and
|
||||
build VM blocks (D3)."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_core.canonical import ToolCall
|
||||
from nisse_core.state import StateStore as CoreStore
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path: Path) -> Path:
|
||||
return tmp_path / "state.db"
|
||||
|
||||
|
||||
def test_a_vm_block_persists_across_reopen(db: Path) -> None:
|
||||
with StateStore(db) as store:
|
||||
store.block_vm("root@vm1", cause="broken", reason="pip crashes", work_item="x")
|
||||
store.block_vm("root@vm1", cause="broken", reason="later", work_item="y")
|
||||
with StateStore(db) as store:
|
||||
[block] = store.vm_blocks().values()
|
||||
assert (block.host, block.reason, block.work_item) == (
|
||||
"root@vm1",
|
||||
"pip crashes", # the first block is kept
|
||||
"x",
|
||||
)
|
||||
assert store.unblock_vm("root@vm1", cause="test") == block
|
||||
assert store.unblock_vm("root@vm1", cause="test") is None
|
||||
assert store.vm_blocks() == {}
|
||||
|
||||
|
||||
def test_a_version_4_store_gains_vm_blocks(db: Path) -> None:
|
||||
with StateStore(db, schema_version=4) as store:
|
||||
store._conn.execute("DROP TABLE vm_blocks") # as a real version 4 store
|
||||
with StateStore(db) as store:
|
||||
assert store.schema_version == 5
|
||||
store.block_vm("root@vm1", cause="broken", reason="r", work_item=None)
|
||||
assert list(store.vm_blocks()) == ["root@vm1"]
|
||||
|
||||
|
||||
def test_the_dev_roles_are_the_dev_stores_own(db: Path) -> None:
|
||||
with StateStore(db) as store:
|
||||
store.transcript("x", "toolchain")
|
||||
assert store.oplog.agent == "nisse-dev"
|
||||
core = CoreStore(db.with_name("core.db"))
|
||||
with core, pytest.raises(ValueError, match="unknown actor"):
|
||||
core.transcript("x", "toolchain")
|
||||
|
||||
|
||||
def test_each_role_is_shown_the_others_as_the_dev_store_says(db: Path) -> None:
|
||||
with StateStore(db) as store:
|
||||
for actor in ("driver", "toolchain"):
|
||||
store.transcript("x", actor).append(
|
||||
ToolCall(call_id=actor, tool="read_file", arguments={"path": "a"})
|
||||
)
|
||||
[seen] = store.transcript("x", "memory").view() # the driver's, told
|
||||
assert seen.actor == "driver" and "read_file(a)" in seen.event.text
|
||||
[own] = store.transcript("x", "driver").view() # not the toolchain's
|
||||
assert own.event.call_id == "driver"
|
||||
@@ -4,13 +4,13 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
)
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_dev.evals import UNMARKED, collect, summarise, to_jsonl
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
|
||||
|
||||
@@ -4,13 +4,16 @@ from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
DelegationCompleted,
|
||||
DelegationStarted,
|
||||
StoredEvent,
|
||||
)
|
||||
from nisse_core.llm import LLMError, RoleMap
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.state import Transcript
|
||||
from nisse_dev.expert import (
|
||||
DEFAULT_BUDGET,
|
||||
BudgetExhausted,
|
||||
@@ -18,9 +21,7 @@ from nisse_dev.expert import (
|
||||
ExpertDesk,
|
||||
parse_answer,
|
||||
)
|
||||
from nisse_dev.llm import LLMError, RoleMap
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore, Transcript
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
ROLES = RoleMap(
|
||||
driver="haiku",
|
||||
|
||||
@@ -6,7 +6,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_dev.git import GitError, GitRepo, HTTPSToken, SSHKey
|
||||
|
||||
|
||||
@@ -170,7 +170,7 @@ class RecordingHost:
|
||||
self.calls: list[dict] = []
|
||||
|
||||
def execute(self, command: str, **kwargs):
|
||||
from nisse_dev.execution import Result
|
||||
from nisse_core.execution import Result
|
||||
|
||||
self.calls.append({"command": command, **kwargs})
|
||||
return Result(0, "", "")
|
||||
|
||||
@@ -8,25 +8,25 @@ from typing import Any
|
||||
import pytest
|
||||
from model_contract import assert_renderable
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
CanonicalEvent,
|
||||
Fact,
|
||||
ToolCall,
|
||||
ToolResult,
|
||||
)
|
||||
from nisse_dev.gitea import Issue
|
||||
from nisse_dev.llm import LLMError
|
||||
from nisse_core.gitea import Issue
|
||||
from nisse_core.llm import LLMError
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.loop.agent import LoopAgent
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import CLOSED, GO_AHEAD, READY, RUNNING, WAITING, loop_of
|
||||
from nisse_dev.loop.reader import command_form
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import ToolOutcome
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
DON = "@don:test"
|
||||
|
||||
@@ -250,7 +250,7 @@ def test_a_go_for_an_earlier_revision_is_not_applied(h: Harness) -> None:
|
||||
h.say("put it in its own module") # changes, so the driver restates rev-2
|
||||
assert loop_of(h.item()).rev == 2
|
||||
# A go sent while rev-1 was on screen, arriving now.
|
||||
from nisse_dev.transport import InboundMessage
|
||||
from nisse_core.transport import InboundMessage
|
||||
|
||||
h.agent.handle(InboundMessage(DON, "go", thread=h.thread()), about=1)
|
||||
assert not loop_of(h.item()).started
|
||||
|
||||
@@ -12,7 +12,7 @@ from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_core.actions import Action
|
||||
from nisse_dev.canonical import ActionVerdict, ToolCall, ToolResult
|
||||
from nisse_core.canonical import ActionVerdict, ToolCall, ToolResult
|
||||
from nisse_dev.loop.actions import classify, commits
|
||||
from nisse_dev.loop.items import ANSWER, loop_of
|
||||
|
||||
|
||||
@@ -14,8 +14,8 @@ from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_core.actions import Action
|
||||
from nisse_dev.canonical import ActionVerdict, ToolCall
|
||||
from nisse_dev.llm import LLMError
|
||||
from nisse_core.canonical import ActionVerdict, ToolCall
|
||||
from nisse_core.llm import LLMError
|
||||
from nisse_dev.loop.classifier import ActionClassifier, Scope
|
||||
from nisse_dev.loop.items import ANSWER, loop_of
|
||||
|
||||
@@ -312,7 +312,7 @@ def test_the_export_script_reads_a_store(tmp_path: Path) -> None:
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
|
||||
store = StateStore(tmp_path / "state.db", oplog=OpLog("t", io.StringIO()))
|
||||
|
||||
@@ -9,8 +9,8 @@ from typing import Any
|
||||
from test_loop_milestones import DON, Forge, ScriptedModel, call, loop_agent
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import Fact, ToolResult
|
||||
from nisse_dev.gitea import Issue
|
||||
from nisse_core.canonical import Fact, ToolResult
|
||||
from nisse_core.gitea import Issue
|
||||
from nisse_dev.loop.agent import FILE_ISSUE
|
||||
from nisse_dev.loop.items import CHAT, GO_AHEAD, MERGE, WAITING, loop_of
|
||||
|
||||
|
||||
@@ -18,16 +18,16 @@ from test_loop_milestones import (
|
||||
)
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.canonical import (
|
||||
AssistantText,
|
||||
DelegationStarted,
|
||||
Fact,
|
||||
ToolCall,
|
||||
ToolResult,
|
||||
)
|
||||
from nisse_core.llm import RoleMap
|
||||
from nisse_dev.config import Profile
|
||||
from nisse_dev.delegation import DELEGATION_INTENT, Caps
|
||||
from nisse_dev.llm import RoleMap
|
||||
from nisse_dev.loop.help import Helpers
|
||||
from nisse_dev.loop.items import ANSWER, MERGE, loop_of
|
||||
|
||||
|
||||
@@ -9,8 +9,8 @@ import pytest
|
||||
from test_loop import DON, RESTATED, FakeGitea, Harness, ScriptedModel, call, issue
|
||||
from test_loop import h as h # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import Fact, ToolCall
|
||||
from nisse_dev.llm import LLMError
|
||||
from nisse_core.canonical import Fact, ToolCall
|
||||
from nisse_core.llm import LLMError
|
||||
from nisse_dev.loop.items import ANSWER, GO_AHEAD, READY, WAITING, loop_of
|
||||
from nisse_dev.loop.reader import GoAheadReader
|
||||
|
||||
@@ -284,7 +284,7 @@ def test_an_unlabelled_issue_waits_for_go_with_no_countdown(h: Harness) -> None:
|
||||
|
||||
|
||||
def _msg(text: str, thread: str) -> Any:
|
||||
from nisse_dev.transport import InboundMessage
|
||||
from nisse_core.transport import InboundMessage
|
||||
|
||||
return InboundMessage(DON, text, thread=thread)
|
||||
|
||||
|
||||
@@ -16,9 +16,9 @@ from test_loop_milestones import RESTATE, Forge, loop_agent, say
|
||||
from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import AssistantText, Fact
|
||||
from nisse_core.canonical import AssistantText, Fact
|
||||
from nisse_core.gitea import Comment
|
||||
from nisse_dev.delegation import Caps
|
||||
from nisse_dev.gitea import Comment
|
||||
from nisse_dev.loop.help import Helpers
|
||||
from nisse_dev.loop.items import GO_AHEAD, MERGE, WAITING, loop_of
|
||||
|
||||
@@ -231,7 +231,7 @@ def test_an_issue_assigned_again_after_its_item_closed_is_taken_again(
|
||||
|
||||
|
||||
def test_abandoning_a_blocked_issue_says_why_it_stays_open(tmp_path: Path) -> None:
|
||||
from nisse_dev.gitea import GiteaError
|
||||
from nisse_core.gitea import GiteaError
|
||||
|
||||
class Strict(FakeGitea):
|
||||
def close_issue(self, repo: str, index: int) -> None:
|
||||
|
||||
@@ -12,17 +12,17 @@ from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
from test_loop_notes import Spy
|
||||
|
||||
from nisse_dev.canonical import AssistantText, Fact
|
||||
from nisse_dev.llm import RoleMap
|
||||
from nisse_dev.memory import Memory
|
||||
from nisse_dev.vault import Vault
|
||||
from nisse_core.canonical import AssistantText, Fact
|
||||
from nisse_core.llm import RoleMap
|
||||
from nisse_core.memory import Memory
|
||||
from nisse_core.vault import Vault
|
||||
|
||||
FOLDER = "memory/nisse-dev"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def memory(tmp_path: Path) -> Memory:
|
||||
return Memory(Vault(tmp_path / "v", {FOLDER: "rw"}), FOLDER)
|
||||
return Memory(Vault(tmp_path / "v", {FOLDER: "rw"}, agent="nisse-dev"), FOLDER)
|
||||
|
||||
|
||||
def facts(world: dict[str, Any]) -> list[str]:
|
||||
|
||||
@@ -10,11 +10,10 @@ from typing import Any
|
||||
import pytest
|
||||
from model_contract import assert_renderable
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.canonical import ToolCall
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.git import GitRepo
|
||||
from nisse_dev.gitea import (
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.canonical import ToolCall
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.gitea import (
|
||||
Approval,
|
||||
Comment,
|
||||
GiteaError,
|
||||
@@ -22,15 +21,16 @@ from nisse_dev.gitea import (
|
||||
PullRequest,
|
||||
ReviewNote,
|
||||
)
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.git import GitRepo
|
||||
from nisse_dev.loop.agent import LoopAgent
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import ANSWER, MERGE, WAITING, loop_of
|
||||
from nisse_dev.loop.milestones import OPEN_PR, Milestones
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.state import StateStore
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
DON = "@don:test"
|
||||
|
||||
|
||||
@@ -18,8 +18,8 @@ from test_loop_milestones import (
|
||||
)
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import ToolCall, ToolResult
|
||||
from nisse_dev.gitea import Issue
|
||||
from nisse_core.canonical import ToolCall, ToolResult
|
||||
from nisse_core.gitea import Issue
|
||||
from nisse_dev.loop.items import MERGE, WAITING, loop_of
|
||||
|
||||
|
||||
|
||||
@@ -14,16 +14,16 @@ import yaml
|
||||
from test_loop_milestones import DON, Forge, ScriptedModel, call, make_world
|
||||
from test_loop_projects import Router
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.config import _parse_projects
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import CHAT, GO_AHEAD, MERGE, PROVISION, loop_of
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
PROPOSAL = call(
|
||||
"propose_project",
|
||||
|
||||
@@ -11,9 +11,9 @@ from test_loop_milestones import DON, RESTATE, ScriptedModel, call, loop_agent
|
||||
from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import ToolResult
|
||||
from nisse_core.canonical import ToolResult
|
||||
from nisse_core.vault import Vault
|
||||
from nisse_dev.loop.items import GO_AHEAD, loop_of
|
||||
from nisse_dev.vault import Vault
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -21,7 +21,7 @@ def vault(tmp_path: Path) -> Vault:
|
||||
root = tmp_path / "vault"
|
||||
(root / "notes").mkdir(parents=True)
|
||||
(root / "notes" / "Style.md").write_text("# Style\n\nCount words with split.\n")
|
||||
return Vault(root, {"memory/nisse-dev": "rw", "notes": "ro"})
|
||||
return Vault(root, {"memory/nisse-dev": "rw", "notes": "ro"}, agent="nisse-dev")
|
||||
|
||||
|
||||
class Spy(ScriptedModel):
|
||||
|
||||
@@ -10,15 +10,15 @@ from typing import Any
|
||||
import pytest
|
||||
from test_loop_milestones import DON, Forge, ScriptedModel, call, git, make_world
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import ANSWER, GO_AHEAD, MERGE, item_for_issue, loop_of
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
REPOS = {"alpha": "don/alpha", "beta": "don/beta"}
|
||||
|
||||
|
||||
@@ -24,8 +24,8 @@ from test_loop_milestones import DON, Forge, call, make_world
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
from test_loop_projects import Router
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.canonical import (
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.canonical import (
|
||||
ActionVerdict,
|
||||
AssistantText,
|
||||
Fact,
|
||||
@@ -34,10 +34,15 @@ from nisse_dev.canonical import (
|
||||
ToolResult,
|
||||
UserMessage,
|
||||
)
|
||||
from nisse_core.execution import HostUnavailable, SSHHost
|
||||
from nisse_core.gitea import Comment, Issue
|
||||
from nisse_core.llm import LLMError, RoleMap
|
||||
from nisse_core.memory import Memory
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InboundMessage, InMemoryTransport
|
||||
from nisse_core.vault import Vault, split_frontmatter
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.delegation import DELEGATION_INTENT, Caps
|
||||
from nisse_dev.execution import HostUnavailable, SSHHost
|
||||
from nisse_dev.gitea import Comment, Issue
|
||||
from nisse_dev.llm import LLMError, RoleMap
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.classifier import ActionClassifier
|
||||
from nisse_dev.loop.driver import Driver
|
||||
@@ -53,12 +58,7 @@ from nisse_dev.loop.items import (
|
||||
WAITING,
|
||||
loop_of,
|
||||
)
|
||||
from nisse_dev.memory import Memory
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InboundMessage, InMemoryTransport
|
||||
from nisse_dev.vault import Vault, split_frontmatter
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
# How many walks, and from which seed: all 48 by default; CI runs fewer on a
|
||||
# PR, from a seed that moves with the run number, so PRs still cover new
|
||||
@@ -360,7 +360,11 @@ class Walk:
|
||||
(vault_root / "notes").mkdir(parents=True, exist_ok=True)
|
||||
(vault_root / "notes" / "Ops.md").write_text("the operator's\n")
|
||||
self.memory = Memory(
|
||||
Vault(vault_root, {"memory/nisse-dev": "rw", "notes": "ro"}),
|
||||
Vault(
|
||||
vault_root,
|
||||
{"memory/nisse-dev": "rw", "notes": "ro"},
|
||||
agent="nisse-dev",
|
||||
),
|
||||
"memory/nisse-dev",
|
||||
)
|
||||
real = SSHHost.execute
|
||||
|
||||
@@ -7,7 +7,8 @@ from typing import Any
|
||||
from test_loop_milestones import DON, Forge, ScriptedModel, call, git, loop_agent
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.gitea import Comment, Issue
|
||||
from nisse_core.gitea import Comment, Issue
|
||||
from nisse_core.transport import InboundMessage, InMemoryTransport
|
||||
from nisse_dev.loop.items import (
|
||||
ANSWER,
|
||||
BLOCKED,
|
||||
@@ -18,7 +19,6 @@ from nisse_dev.loop.items import (
|
||||
item_for_issue,
|
||||
loop_of,
|
||||
)
|
||||
from nisse_dev.transport import InboundMessage, InMemoryTransport
|
||||
|
||||
|
||||
def two_issues(forge: Forge) -> None:
|
||||
|
||||
@@ -8,7 +8,7 @@ from typing import Any
|
||||
|
||||
from test_loop import DON, RESTATED, FakeGitea, Harness, ScriptedModel, call, issue
|
||||
|
||||
from nisse_dev.canonical import AssistantText
|
||||
from nisse_core.canonical import AssistantText
|
||||
from nisse_dev.loop.items import ANSWER, GO_AHEAD, WAITING, loop_of
|
||||
|
||||
|
||||
|
||||
@@ -26,18 +26,18 @@ from test_loop_milestones import (
|
||||
from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_dev.canonical import Fact, ToolResult
|
||||
from nisse_dev.execution import HostKeyChanged
|
||||
from nisse_dev.gitea import Issue
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.canonical import Fact, ToolResult
|
||||
from nisse_core.execution import HostKeyChanged
|
||||
from nisse_core.gitea import Issue
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.loop import preflight
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import ANSWER, GO_AHEAD, WAITING, loop_of
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
PYTHON = {"name": "python", "version_command": "python3 --version"}
|
||||
|
||||
@@ -194,7 +194,7 @@ def test_what_was_installed_or_is_missing_is_said(
|
||||
def test_an_unfinished_step_is_said_and_work_goes_on(
|
||||
world: dict[str, Any], project: str
|
||||
) -> None:
|
||||
from nisse_dev.canonical import AssistantText
|
||||
from nisse_core.canonical import AssistantText
|
||||
|
||||
prose = [AssistantText(text="hmm", model="m")]
|
||||
model = ScriptedModel(prose, prose, RESTATE)
|
||||
|
||||
@@ -19,14 +19,14 @@ from test_loop_milestones import (
|
||||
from test_loop_milestones import item as the_item
|
||||
from test_loop_milestones import world as world # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.cancel import Cancel
|
||||
from nisse_core.cancel import Cancel
|
||||
from nisse_core.oplog import OpLog
|
||||
from nisse_core.transport import InMemoryTransport
|
||||
from nisse_core.worker import InlineWorker
|
||||
from nisse_dev.loop.agent import LoopAgent, Project
|
||||
from nisse_dev.loop.driver import Driver
|
||||
from nisse_dev.loop.items import MERGE, loop_of
|
||||
from nisse_dev.oplog import OpLog
|
||||
from nisse_dev.tools import Workspace
|
||||
from nisse_dev.transport import InMemoryTransport
|
||||
from nisse_dev.worker import InlineWorker
|
||||
|
||||
CLOSE = [call("close_item", summary="added and merged")]
|
||||
|
||||
@@ -169,7 +169,7 @@ def test_unprotected_means_the_operator_merges(world: dict[str, Any]) -> None:
|
||||
def test_a_refused_merge_is_said_once_and_left_to_the_operator(
|
||||
world: dict[str, Any],
|
||||
) -> None:
|
||||
from nisse_dev.gitea import GiteaError
|
||||
from nisse_core.gitea import GiteaError
|
||||
|
||||
forge: Forge = world["forge"]
|
||||
agent, transport = to_review(world)
|
||||
|
||||
@@ -19,9 +19,9 @@ from test_loop_toolchain import (
|
||||
)
|
||||
from test_loop_toolchain import project as project # noqa: PLC0414 -- the fixture
|
||||
|
||||
from nisse_dev.canonical import ToolResult
|
||||
from nisse_dev.execution import HostKeyChanged, HostUnavailable
|
||||
from nisse_dev.gitea import Issue
|
||||
from nisse_core.canonical import ToolResult
|
||||
from nisse_core.execution import HostKeyChanged, HostUnavailable
|
||||
from nisse_core.gitea import Issue
|
||||
from nisse_dev.loop.items import GO_AHEAD, READY, loop_of
|
||||
|
||||
RETRY, SETTLE = 300.0, 180.0
|
||||
|
||||
@@ -5,7 +5,7 @@ import uuid
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.execution import SSHHost
|
||||
from nisse_core.execution import SSHHost
|
||||
from nisse_dev.loop import preflight
|
||||
from nisse_dev.loop.preflight import Tool
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from nisse_dev.execution import REDACTED, SSHHost
|
||||
from nisse_core.execution import REDACTED, SSHHost
|
||||
from nisse_dev.tools import EXECUTION_TOOLS, MAX_OUTPUT_CHARS, Workspace
|
||||
|
||||
|
||||
|
||||
@@ -14,12 +14,12 @@ import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_core.catalog import ModelCatalog
|
||||
from nisse_core.envfile import load_env_file
|
||||
from nisse_dev.bench.results import append, load, render, results_file, summarise
|
||||
from nisse_dev.bench.runner import agent_commit, run_task
|
||||
from nisse_dev.bench.tasks import load_tasks
|
||||
from nisse_dev.catalog import ModelCatalog
|
||||
from nisse_dev.config import AgentConfig, ConfigError
|
||||
from nisse_dev.envfile import load_env_file
|
||||
from nisse_dev.loop.build import build_host, make_llm_clients
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
|
||||
@@ -23,8 +23,8 @@ from pathlib import Path
|
||||
sys.path.insert(0, "tests/integration")
|
||||
from e2e_support import OWNER, delete_repo, make_repo, operator_gitea # noqa: E402
|
||||
|
||||
from nisse_dev.envfile import load_env_file # noqa: E402
|
||||
from nisse_dev.gitea import GiteaClient # noqa: E402
|
||||
from nisse_core.envfile import load_env_file # noqa: E402
|
||||
from nisse_core.gitea import GiteaClient # noqa: E402
|
||||
|
||||
action, e2e = sys.argv[1], Path(sys.argv[2])
|
||||
url = "https://gitea.dongar.net"
|
||||
|
||||
+1
-1
@@ -133,7 +133,7 @@ cmd_logs() { journalctl -u "$UNIT" -f -o cat; }
|
||||
# The state store, read with the image's own code.
|
||||
dump() {
|
||||
docker run --rm --user 0 -v "$E2E:$E2E" --entrypoint python "$IMAGE" \
|
||||
-m nisse_dev.dump "$@"
|
||||
-m nisse_core.dump "$@"
|
||||
}
|
||||
|
||||
cmd_collect() {
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# The throwaway sshd the tests need (nisse_dev/testing.py), in a CI job.
|
||||
# The throwaway sshd the tests need (nisse_core/testing.py), in a CI job.
|
||||
# Skipped when the job's image already has it. Otherwise apt fetches from a
|
||||
# kernel.org mirror rather than archive.ubuntu.com, which has been slow
|
||||
# enough (a few KB/s) to add minutes to every job.
|
||||
|
||||
@@ -19,7 +19,7 @@ import sys
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
|
||||
from nisse_dev.canonical import StoredEvent, from_payload
|
||||
from nisse_core.canonical import StoredEvent, from_payload
|
||||
from nisse_dev.evals import classifier_cases
|
||||
|
||||
|
||||
|
||||
Loaded 100 of 119 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user