Compare commits
30
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2352a84004 | ||
|
|
9362a134c9 | ||
|
|
b8ba683595 | ||
|
|
d8d37b5441 | ||
|
|
af9bdaac8d | ||
|
|
53ed11f9c4 | ||
|
|
1c8e7e35fc | ||
|
|
f4a94281ef | ||
|
|
1f292793c1 | ||
|
|
f03c201729 | ||
|
|
886cb80583 | ||
|
|
846e5494ca | ||
|
|
8f365263c4 | ||
|
|
acee8b96f0 | ||
|
|
d508e9a9b7 | ||
|
|
affdb9e8aa | ||
|
|
914bc26221 | ||
|
|
3b1b84162a | ||
|
|
3117565928 | ||
|
|
4d0b24794b | ||
|
|
3e664984ac | ||
|
|
4025ac7875 | ||
|
|
1ebd2a9988 | ||
|
|
ad45e33517 | ||
|
|
8dd5e3eb15 | ||
|
|
2071e6ab6d | ||
|
|
e04d786840 | ||
|
|
bacf65dbe2 | ||
|
|
999a38a705 | ||
|
|
0fd0b2e4a5 |
No files matched your search
@@ -25,3 +25,12 @@ NISSE_MATRIX_ROOM_ID=!roomid:example
|
|||||||
# Dedicated room for the live integration test; never the production room.
|
# Dedicated room for the live integration test; never the production room.
|
||||||
NISSE_MATRIX_TEST_ROOM_ID=!testroomid:example
|
NISSE_MATRIX_TEST_ROOM_ID=!testroomid:example
|
||||||
NISSE_MATRIX_OPERATORS=@you:example
|
NISSE_MATRIX_OPERATORS=@you:example
|
||||||
|
|
||||||
|
# --- Gitea (B2): service account for branches, PRs, and comment intake ---
|
||||||
|
NISSE_GITEA_URL=https://gitea.example
|
||||||
|
NISSE_GITEA_TOKEN=your-scoped-service-account-token
|
||||||
|
# Sandbox repo for live tests; never the real project repo.
|
||||||
|
NISSE_GITEA_TEST_REPO=owner/sandbox-repo
|
||||||
|
# Gitea logins allowed to drive the agent via PR comments (A4 per-channel
|
||||||
|
# allowlist; a Matrix MXID and a Gitea login are different names).
|
||||||
|
NISSE_GITEA_OPERATORS=your-gitea-login
|
||||||
@@ -1,7 +1,14 @@
|
|||||||
name: CI
|
name: CI
|
||||||
on:
|
on:
|
||||||
|
# Only main on push. A PR branch is already covered by the pull_request
|
||||||
|
# event, and running both fired two identical suites concurrently — which
|
||||||
|
# doubled the LLM tokens the live tests spend and tripped the proxy's
|
||||||
|
# per-minute limit.
|
||||||
push:
|
push:
|
||||||
|
branches: [main]
|
||||||
pull_request:
|
pull_request:
|
||||||
|
# Manual re-run, e.g. after rotating a credential or raising a budget cap.
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
checks:
|
checks:
|
||||||
@@ -39,4 +46,7 @@ jobs:
|
|||||||
NISSE_MATRIX_USER_ID: ${{ secrets.NISSE_MATRIX_USER_ID }}
|
NISSE_MATRIX_USER_ID: ${{ secrets.NISSE_MATRIX_USER_ID }}
|
||||||
NISSE_MATRIX_TOKEN: ${{ secrets.NISSE_MATRIX_TOKEN }}
|
NISSE_MATRIX_TOKEN: ${{ secrets.NISSE_MATRIX_TOKEN }}
|
||||||
NISSE_MATRIX_TEST_ROOM_ID: ${{ secrets.NISSE_MATRIX_TEST_ROOM_ID }}
|
NISSE_MATRIX_TEST_ROOM_ID: ${{ secrets.NISSE_MATRIX_TEST_ROOM_ID }}
|
||||||
|
NISSE_GITEA_URL: ${{ secrets.NISSE_GITEA_URL }}
|
||||||
|
NISSE_GITEA_TOKEN: ${{ secrets.NISSE_GITEA_TOKEN }}
|
||||||
|
NISSE_GITEA_TEST_REPO: ${{ secrets.NISSE_GITEA_TEST_REPO }}
|
||||||
run: uv run pytest -m integration -rs
|
run: uv run pytest -m integration -rs
|
||||||
@@ -175,7 +175,9 @@ Requirements:
|
|||||||
task. The agent has no path to approve itself.
|
task. The agent has no path to approve itself.
|
||||||
- **Approval sources.** A gate waits on an approval event from any registered
|
- **Approval sources.** A gate waits on an approval event from any registered
|
||||||
**`ApprovalSource`** — the Matrix transport (A1), the Gitea comment poller (B2), or
|
**`ApprovalSource`** — the Matrix transport (A1), the Gitea comment poller (B2), or
|
||||||
the Vikunja comment stream (C4). First valid approval wins and is consumed; the
|
the Vikunja comment stream (C4). **The knowledge store (A17) is never an approval
|
||||||
|
source**, and never an instruction channel: a wiki page has neither the identity
|
||||||
|
guarantees nor the revision binding an approval requires. First valid approval wins and is consumed; the
|
||||||
gate records which channel satisfied it, and the orchestrator echoes the outcome to
|
gate records which channel satisfied it, and the orchestrator echoes the outcome to
|
||||||
the other channels so the records converge. Each source authenticates the approver
|
the other channels so the records converge. Each source authenticates the approver
|
||||||
against a per-channel operator allowlist (Matrix MXID, Gitea username, Vikunja
|
against a per-channel operator allowlist (Matrix MXID, Gitea username, Vikunja
|
||||||
@@ -333,6 +335,7 @@ The durable record is a **canonical event log** in SQLite, not a provider-shaped
|
|||||||
| `delegation_started` | brief, target role, resolved model, caps |
|
| `delegation_started` | brief, target role, resolved model, caps |
|
||||||
| `delegation_completed` | handback note, artifacts, tokens, duration, exit status |
|
| `delegation_completed` | handback note, artifacts, tokens, duration, exit status |
|
||||||
| `compaction` | summary, producing model, range of events replaced |
|
| `compaction` | summary, producing model, range of events replaced |
|
||||||
|
| `knowledge_retrieved` | document text as read, document ID, revision, retrieved-at (A17) |
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
- **Canonical tool-call IDs are minted locally.** Provider IDs are mapped at the
|
- **Canonical tool-call IDs are minted locally.** Provider IDs are mapped at the
|
||||||
@@ -507,6 +510,167 @@ additional cost is marking outcomes as good or bad.
|
|||||||
Without this, "best model this week" is decided by benchmark chatter, and swapping the
|
Without this, "best model this week" is decided by benchmark chatter, and swapping the
|
||||||
frontier tier becomes something to avoid because the effect is unmeasurable. With it, a
|
frontier tier becomes something to avoid because the effect is unmeasurable. With it, a
|
||||||
new model is a config change and a re-run.
|
new model is a config change and a re-run.
|
||||||
|
### A17. Knowledge Store (Outline)
|
||||||
|
|
||||||
|
**Purpose:** durable knowledge that outlives a single task — an agent-maintained
|
||||||
|
**world model** (what infrastructure exists, how it is arranged) and, later,
|
||||||
|
operator-authored **procedures**. Distinct from A2, which holds *task* state: A2
|
||||||
|
is the agent's memory of what it is doing, A17 is its memory of what the world
|
||||||
|
looks like. Neither is a fallback for the other.
|
||||||
|
|
||||||
|
Part D describes the service; this defines the contract.
|
||||||
|
|
||||||
|
**Layout — sharing is read access, never write access.**
|
||||||
|
- Every document has **exactly one writer**: one agent, or the operator. Sharing
|
||||||
|
governs who may *read*. Concurrent writers therefore never exist, which
|
||||||
|
disposes of the last-write-wins objection Part D raises, without needing
|
||||||
|
distributed-write machinery.
|
||||||
|
- **Agent-private** space per agent: its own world model.
|
||||||
|
- **Shared** space: readable by the operator and designated agents. Default
|
||||||
|
membership is **operator + `nisse-ops`**. `nisse-dev` gets private space only —
|
||||||
|
B3 seals it from RFC1918 precisely so it cannot reach the NAS, Proxmox, or the
|
||||||
|
UDM, and a shared network-architecture document is a map of exactly those
|
||||||
|
things. Cross-agent read access is justified per collection, never assumed.
|
||||||
|
- The dev agent's project conventions live in the **repo it is working on**, not
|
||||||
|
here: versioned with the code, reviewed in the PR, and impossible to
|
||||||
|
desynchronise from the project.
|
||||||
|
|
||||||
|
**Agent-owned knowledge is always a cache, never authority.** An agent may rely on
|
||||||
|
it to know *what exists and where to look*; it must confirm a specific fact
|
||||||
|
against the live system *before acting on it*. Hard requirement for anything in
|
||||||
|
C2's confirm-first tier; advisory for read-only and reversible actions.
|
||||||
|
|
||||||
|
**Enforcement is structural, because a prompt is not enforcement.** "Treat this as
|
||||||
|
a cache" is exactly the advisory guarantee A4 and A12 reject, and the model cannot
|
||||||
|
be prevented from *believing* a fact it has read. The enforceable goal is
|
||||||
|
narrower: **an unverified belief must not become an irreversible action.**
|
||||||
|
1. **The store decides what it serves.** Facts carry provenance and an observed-at
|
||||||
|
timestamp; retrieval marks or withholds anything past a staleness bound for
|
||||||
|
high-consequence classes.
|
||||||
|
2. **Verification as a precondition, checked against the log.** For confirm-first
|
||||||
|
actions, refuse unless a verifying observation exists *in this task's canonical
|
||||||
|
log*, recorded after the task began, naming the action's target. Inspectable
|
||||||
|
without asking the model anything — the same move as A12's observable triggers.
|
||||||
|
Approximate by nature, so build it for the confirm-first tier only.
|
||||||
|
3. **Provenance at the human gate** — the primary reliance, because C2 already
|
||||||
|
stops these actions for confirmation. The prompt states evidence quality:
|
||||||
|
"target recalled from the world model, observed 2026-08-02, not verified this
|
||||||
|
session" versus "confirmed by `zfs list` two minutes ago". The operator is
|
||||||
|
already the authority there; this makes the risk legible rather than invisible.
|
||||||
|
4. **Audit, not prevention, for the cheap tiers.** Log which facts were retrieved
|
||||||
|
and which actions followed, so a wrong fact is traceable afterwards.
|
||||||
|
|
||||||
|
Residual risk, stated plainly: a model can retype a fact from context with no
|
||||||
|
retrieval event, and no mechanism here catches that. That is why the world model
|
||||||
|
stays out of the authority path entirely.
|
||||||
|
|
||||||
|
**Never an instruction channel, and never an approval channel.**
|
||||||
|
- **Task instructions never come from the knowledge store.** Work is driven by
|
||||||
|
Vikunja tickets (C4) and operator commands (B2). An agent must never read the
|
||||||
|
store looking for work to do.
|
||||||
|
- **Approvals never come from the knowledge store.** The registered
|
||||||
|
`ApprovalSource`s are Matrix (A1), Gitea (B2), and Vikunja (C4). A wiki page is
|
||||||
|
not one and must not become one.
|
||||||
|
- An agent must never write here intending to cause another agent to act. This
|
||||||
|
extends Part D's rejection of the store as a message bus: the erosion path is
|
||||||
|
one well-meaning "note for the other agent" at a time.
|
||||||
|
|
||||||
|
**Corrections flow through transport, not through direct edits.** The operator
|
||||||
|
corrects a fact by telling the agent, which is strictly better than editing the
|
||||||
|
page by hand: a hand edit is invisible to the agent's record, whereas an
|
||||||
|
instruction arrives on an authenticated channel from an allowlisted identity, is
|
||||||
|
recorded in the canonical log, and can be reconciled against what the agent
|
||||||
|
believed. Corrections are stored **verbatim with attribution**, never paraphrased —
|
||||||
|
a model rewriting a precise fact loses precision invisibly.
|
||||||
|
- **Break-glass:** the operator keeps write permission. The procedure reuses Part
|
||||||
|
F's idle flag — wait for idle, stop the agent, edit freely, restart. Single
|
||||||
|
writer holds trivially while the agent is down, and A2/A3 make stopping safe.
|
||||||
|
- Because that permission exists, an agent re-reads and compares the revision
|
||||||
|
immediately before writing, and re-merges rather than clobbering if it moved.
|
||||||
|
A safety net for the rare edit, not a designed workflow.
|
||||||
|
|
||||||
|
**Canonical log representation (A10).** Retrieved content is **snapshotted into the
|
||||||
|
log** — the text, plus document id and revision for provenance — never referenced
|
||||||
|
by handle alone: A10 forbids a provider handle being the only reference to content,
|
||||||
|
and these documents are mutable. Snapshot once per task at first use, not per turn.
|
||||||
|
**A3 consequence:** a task resumed across a restart carries the old snapshot, so
|
||||||
|
boot reconciliation re-fetches snapshotted knowledge and flags divergence rather
|
||||||
|
than silently continuing on a stale copy.
|
||||||
|
|
||||||
|
**Retrieval is structural first, discretionary second** — the A12/A14 pattern.
|
||||||
|
- Procedures are **bound to C2 action classes**: the policy taxonomy and the
|
||||||
|
procedure index are the same list. Binding is deterministic and auditable
|
||||||
|
("which procedure governed this action?" has an exact answer), and costs nothing
|
||||||
|
extra because the taxonomy must exist anyway.
|
||||||
|
- A model-invoked **search** exists *in addition* for discovery, never as a
|
||||||
|
replacement for the bound retrieval — a procedure the model may skip does not
|
||||||
|
constrain it.
|
||||||
|
|
||||||
|
**Outage posture is asymmetric.** Unlike B2's poller, "never fails a task" does not
|
||||||
|
generalise here.
|
||||||
|
- **Write failure never blocks:** retry, or record locally and reconcile.
|
||||||
|
- **Read failure for a procedure bound to a confirm-first action gates.**
|
||||||
|
Proceeding is not a delayed reaction; it is acting unconstrained on precisely the
|
||||||
|
operation someone wrote a procedure for.
|
||||||
|
- Read failure for read-only or reversible actions proceeds, logging the gap.
|
||||||
|
|
||||||
|
**Accepted single point of failure.** Once the agent is the sole author and Outline
|
||||||
|
the only home, losing Outline loses the world model. No local mirror: infrastructure
|
||||||
|
facts are re-derivable by observation, which is the same premise the cache rule
|
||||||
|
rests on.
|
||||||
|
|
||||||
|
**Deferred:** the operator-writable procedures collection. The interface is defined
|
||||||
|
here; the content and the C2 binding list arrive with the policy taxonomy (Part H).
|
||||||
|
|
||||||
|
**Concrete mapping onto Outline.** Observed in the running instance: a **collection**
|
||||||
|
carries per-user permissions with three tiers — visible-to (read), read-write, and
|
||||||
|
manager. Permissions are collection-scoped, not per-document.
|
||||||
|
- Therefore **one collection per writing principal**, with read membership granted to
|
||||||
|
everyone else who needs it: `nisse-ops-world` (write: `nisse-ops`),
|
||||||
|
`nisse-dev-world` (write: `nisse-dev`), `procedures` (write: operator).
|
||||||
|
- **What this enforces, precisely:** no *other agent* can write a given agent's world
|
||||||
|
model — the collision that would otherwise be silent and unattributable. It does not
|
||||||
|
enforce operator-versus-agent exclusion, because the operator holds manager on their
|
||||||
|
own wiki and can always grant themselves write. That half is procedural: the
|
||||||
|
break-glass routine stops the agents first, so concurrent writers do not exist while
|
||||||
|
editing.
|
||||||
|
- **Assign per user, not per group.** Outline groups appear to be internal to Outline
|
||||||
|
rather than federated with LLDAP (A8), so using them would put membership in a second
|
||||||
|
place that nothing syncs. With three principals the manual mapping is trivial and
|
||||||
|
obvious; revisit only if the cast grows.
|
||||||
|
- **Each agent's API token must belong to that agent's own Outline user.** Outline API
|
||||||
|
keys are per-user, so the token *is* the identity every permission above evaluates
|
||||||
|
against. A token minted from the operator's account — or an admin's — silently voids
|
||||||
|
the entire boundary while appearing to work.
|
||||||
|
- **Identify collections and documents by ID, from config and state.** Collection IDs
|
||||||
|
come from env (`NISSE_OUTLINE_*_COLLECTION`), never discovered from "whatever the
|
||||||
|
token can write": a permission change must not silently redefine what an agent
|
||||||
|
considers its own. Document IDs are recorded in the state store (A2) when created and
|
||||||
|
used for updates, which also makes writing idempotent — a retry after a crash updates
|
||||||
|
the document instead of creating a second one.
|
||||||
|
- **Revision reads, not conditional writes.** Because break-glass stops the agents,
|
||||||
|
compare-and-swap is unnecessary; what is needed is the ability to *observe* that a
|
||||||
|
document changed since the agent last read it, feeding the A3 divergence check.
|
||||||
|
Whether Outline exposes a revision identifier or only a modified timestamp is
|
||||||
|
unverified — either satisfies this.
|
||||||
|
|
||||||
|
#### Considered and deferred: Outline's MCP server
|
||||||
|
|
||||||
|
Outline can expose search, read, create and edit over MCP (remote, Streamable HTTP,
|
||||||
|
enabled workspace-wide). **Not adopted for the retrieval path**, because A17's
|
||||||
|
enforcement depends on this library sitting *between* the model and the store: staleness
|
||||||
|
marking, snapshotting into the canonical log, binding procedures to action classes, and
|
||||||
|
gating a failed confirm-first read all happen to a retrieval before the model sees it.
|
||||||
|
MCP's purpose is the opposite — direct model access — and a procedure the model may
|
||||||
|
decline to fetch does not constrain it (A12).
|
||||||
|
|
||||||
|
It remains a reasonable candidate for the *discretionary* search tool later, and unlike
|
||||||
|
the vendor agent SDKs A13 rejects, MCP is a neutral standard rather than lock-in: one
|
||||||
|
client would serve many servers if others are ever added. Before adopting it, verify
|
||||||
|
that it honours the key owner's **collection** permissions rather than mere workspace
|
||||||
|
membership, and that the workspace-level toggle does not force broader exposure than
|
||||||
|
intended.
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -550,6 +714,25 @@ PR_OPEN
|
|||||||
- *Outbound:* `git` + Gitea API to push branches and open PRs, via a dedicated
|
- *Outbound:* `git` + Gitea API to push branches and open PRs, via a dedicated
|
||||||
Gitea **service account** with a scoped token. Record any runnable test service's
|
Gitea **service account** with a scoped token. Record any runnable test service's
|
||||||
URL/port in the PR description.
|
URL/port in the PR description.
|
||||||
|
- **The project repo's default branch must be protected, and this is what makes
|
||||||
|
the PR gate real.** The service account needs write access to push branches —
|
||||||
|
and in Gitea, *write access includes merging pull requests*. Repository
|
||||||
|
ownership does not help: an account that merely collaborates can still merge,
|
||||||
|
and one that owns the repo can merge and delete it. So without branch
|
||||||
|
protection the agent can merge its own PR, and the review gate holds only
|
||||||
|
because the agent's code does not call the merge endpoint. That is precisely
|
||||||
|
the advisory guarantee A4 exists to rule out.
|
||||||
|
- Required on the default branch: **block direct pushes** by the agent
|
||||||
|
account, and **restrict merging** to the operator (a merge whitelist, or
|
||||||
|
required approvals that the agent's own account cannot satisfy).
|
||||||
|
- Feature-branch pushes are unaffected, so the agent's normal workflow is
|
||||||
|
unchanged.
|
||||||
|
- **Verify it rather than assume it.** The property is observable from
|
||||||
|
outside: the agent's own token must be refused when it attempts to merge
|
||||||
|
its own PR and when it pushes to the default branch. Assert both, so that
|
||||||
|
relaxing the protection later fails a test instead of silently removing
|
||||||
|
the gate. (Discovered the hard way — before protection was configured, the
|
||||||
|
agent's token merged its own PR successfully.)
|
||||||
- *Inbound:* react to PR comments, commits, pushes. **Poll** the Gitea API
|
- *Inbound:* react to PR comments, commits, pushes. **Poll** the Gitea API
|
||||||
(15–30s) rather than take webhooks, to keep the VM outbound-only. Intake is
|
(15–30s) rather than take webhooks, to keep the VM outbound-only. Intake is
|
||||||
structured JSON — never via a chat bridge.
|
structured JSON — never via a chat bridge.
|
||||||
@@ -670,6 +853,10 @@ Requirements:
|
|||||||
back to Outline: searchable, off-box, and an extra externalized record that
|
back to Outline: searchable, off-box, and an extra externalized record that
|
||||||
survives volume loss.
|
survives volume loss.
|
||||||
- Access via Outline **REST API**, dedicated service account, token from Bitwarden.
|
- Access via Outline **REST API**, dedicated service account, token from Bitwarden.
|
||||||
|
- **The library contract is A17**, which settles what this section left implicit:
|
||||||
|
per-agent private space plus a shared space, one writer per document, agent-owned
|
||||||
|
knowledge treated as a cache rather than authority, and the structural means of
|
||||||
|
keeping it that way.
|
||||||
|
|
||||||
**Deliberately rejected:** using a shared notes store (Outline *or* an Obsidian vault)
|
**Deliberately rejected:** using a shared notes store (Outline *or* an Obsidian vault)
|
||||||
as an **agent-to-agent message bus**. It's unstructured, race-prone (last-write-wins,
|
as an **agent-to-agent message bus**. It's unstructured, race-prone (last-write-wins,
|
||||||
@@ -748,7 +935,8 @@ each agent is built — not standing up the service itself.
|
|||||||
here). Crash-atomic writes and reconcile-on-boot must be in place first.
|
here). Crash-atomic writes and reconcile-on-boot must be in place first.
|
||||||
10. **Extract `nisse-core`**; build the **Sysadmin Agent** (`nisse-ops`): policy
|
10. **Extract `nisse-core`**; build the **Sysadmin Agent** (`nisse-ops`): policy
|
||||||
engine (C2), SSO + scoped browser (A8/A6), secrets (A7), Vikunja control (C4),
|
engine (C2), SSO + scoped browser (A8/A6), secrets (A7), Vikunja control (C4),
|
||||||
Outline runbooks (D), scheduled checks, alert intake.
|
knowledge store (A17/D) — world model first, operator procedures when the
|
||||||
|
C2 taxonomy they bind to exists — scheduled checks, alert intake.
|
||||||
- With the monorepo layout (Part H, decided), extraction is a **package split**
|
- With the monorepo layout (Part H, decided), extraction is a **package split**
|
||||||
within one tree: move code behind a package boundary, no release machinery
|
within one tree: move code behind a package boundary, no release machinery
|
||||||
required.
|
required.
|
||||||
@@ -816,6 +1004,7 @@ nisse-core/ # shared library (extracted after nisse-dev works)
|
|||||||
secrets/ # Bitwarden machine-account client (A7)
|
secrets/ # Bitwarden machine-account client (A7)
|
||||||
sso/ # LLDAP identity + auth path (A8)
|
sso/ # LLDAP identity + auth path (A8)
|
||||||
logging/ # structured operational logs (A9)
|
logging/ # structured operational logs (A9)
|
||||||
|
knowledge/ # Outline client, world model, procedures (A17)
|
||||||
|
|
||||||
nisse-dev/ # app: sealed VM (Part B)
|
nisse-dev/ # app: sealed VM (Part B)
|
||||||
workflow (arch/milestone gating), Gitea PR layer, allow-all sandbox
|
workflow (arch/milestone gating), Gitea PR layer, allow-all sandbox
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
"""Plan externalisation and Gitea-backed approvals (spec A4, Part H).
|
||||||
|
|
||||||
|
Plans are posted as comments on a per-project Gitea issue. The approval
|
||||||
|
comment — its author, timestamp, and revision tag — is the durable off-box
|
||||||
|
record, authenticated by Gitea's own auth. The state store keeps only a
|
||||||
|
pointer to it, so boot reconciliation can re-derive gate state from Gitea
|
||||||
|
even after the state volume is lost: external truth wins.
|
||||||
|
|
||||||
|
Approvals arriving here bind to a revision exactly as they do over transport
|
||||||
|
(A4). The first valid approval on any channel wins; the orchestrator echoes
|
||||||
|
the outcome to the other channel so the records converge.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from nisse_dev.gitea import Comment, GiteaClient, GiteaError
|
||||||
|
|
||||||
|
APPROVAL_HINT = (
|
||||||
|
"Reply `approve {tag}` to approve this revision, or comment with feedback "
|
||||||
|
"to reject it."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PlanPost:
|
||||||
|
issue: int
|
||||||
|
comment_id: int
|
||||||
|
|
||||||
|
|
||||||
|
class PlanBoard:
|
||||||
|
"""The Gitea side of plan gating: one issue per project, one comment per
|
||||||
|
plan revision."""
|
||||||
|
|
||||||
|
def __init__(self, client: GiteaClient, repo: str) -> None:
|
||||||
|
self._client = client
|
||||||
|
self._repo = repo
|
||||||
|
|
||||||
|
@property
|
||||||
|
def repo(self) -> str:
|
||||||
|
return self._repo
|
||||||
|
|
||||||
|
def open_project_issue(self, project: str, task_id: str) -> int:
|
||||||
|
return self._client.create_issue(
|
||||||
|
self._repo,
|
||||||
|
title=f"nisse-dev: {project}",
|
||||||
|
body=(
|
||||||
|
f"Plan gate for project `{project}` (task `{task_id}`).\n\n"
|
||||||
|
"Each plan revision is posted here as a comment. Approving a "
|
||||||
|
"revision by comment is equivalent to approving it in Matrix; "
|
||||||
|
"the first valid approval on either channel wins."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def post_plan(self, issue: int, gate: str, rev: int, plan: str) -> PlanPost:
|
||||||
|
tag = f"rev-{rev}"
|
||||||
|
body = "\n".join(
|
||||||
|
[
|
||||||
|
f"## Plan for `{gate}` ({tag})",
|
||||||
|
"",
|
||||||
|
plan,
|
||||||
|
"",
|
||||||
|
"---",
|
||||||
|
APPROVAL_HINT.format(tag=tag),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
comment = self._client.post_comment(self._repo, issue, body)
|
||||||
|
return PlanPost(issue=issue, comment_id=comment.id)
|
||||||
|
|
||||||
|
def echo(self, issue: int, text: str) -> None:
|
||||||
|
"""Mirror an outcome decided on another channel, so the Gitea record
|
||||||
|
does not silently disagree with Matrix."""
|
||||||
|
try:
|
||||||
|
self._client.post_comment(self._repo, issue, text)
|
||||||
|
except GiteaError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def comments_since(self, issue: int, after_id: int) -> list[Comment]:
|
||||||
|
return [
|
||||||
|
c for c in self._client.list_comments(self._repo, issue) if c.id > after_id
|
||||||
|
]
|
||||||
@@ -8,6 +8,7 @@ arrives over transport. Every position change is checkpointed through the state
|
|||||||
store before the loop continues, so a restart lands mid-gate, not mid-air.
|
store before the loop continues, so a restart lands mid-gate, not mid-air.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import threading
|
||||||
import uuid
|
import uuid
|
||||||
from collections.abc import Iterable, Mapping
|
from collections.abc import Iterable, Mapping
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
@@ -15,6 +16,8 @@ from typing import Any, Protocol
|
|||||||
|
|
||||||
from nisse_dev.canonical import GateOpened, PlanSubmitted, UserMessage
|
from nisse_dev.canonical import GateOpened, PlanSubmitted, UserMessage
|
||||||
from nisse_dev.gates import Approved, Gate, Ignored, PlanRevision, Rejected
|
from nisse_dev.gates import Approved, Gate, Ignored, PlanRevision, Rejected
|
||||||
|
from nisse_dev.git import GitError
|
||||||
|
from nisse_dev.gitea import GiteaError
|
||||||
from nisse_dev.llm import RoleMap
|
from nisse_dev.llm import RoleMap
|
||||||
from nisse_dev.reconcile import Reconciler, reconcile_on_boot
|
from nisse_dev.reconcile import Reconciler, reconcile_on_boot
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore
|
||||||
@@ -26,6 +29,7 @@ AWAIT_ARCH_SIGNOFF = "AWAIT_ARCH_SIGNOFF"
|
|||||||
MILESTONE_PLANNING = "MILESTONE_PLANNING"
|
MILESTONE_PLANNING = "MILESTONE_PLANNING"
|
||||||
AWAIT_MILESTONE_SIGNOFF = "AWAIT_MILESTONE_SIGNOFF"
|
AWAIT_MILESTONE_SIGNOFF = "AWAIT_MILESTONE_SIGNOFF"
|
||||||
EXECUTING = "EXECUTING"
|
EXECUTING = "EXECUTING"
|
||||||
|
PR_OPEN = "PR_OPEN"
|
||||||
AWAIT_INPUT = "AWAIT_INPUT"
|
AWAIT_INPUT = "AWAIT_INPUT"
|
||||||
|
|
||||||
TRANSITIONS = {
|
TRANSITIONS = {
|
||||||
@@ -36,6 +40,12 @@ TRANSITIONS = {
|
|||||||
(MILESTONE_PLANNING, AWAIT_MILESTONE_SIGNOFF),
|
(MILESTONE_PLANNING, AWAIT_MILESTONE_SIGNOFF),
|
||||||
(AWAIT_MILESTONE_SIGNOFF, MILESTONE_PLANNING),
|
(AWAIT_MILESTONE_SIGNOFF, MILESTONE_PLANNING),
|
||||||
(AWAIT_MILESTONE_SIGNOFF, EXECUTING),
|
(AWAIT_MILESTONE_SIGNOFF, EXECUTING),
|
||||||
|
(EXECUTING, PR_OPEN),
|
||||||
|
# PR acceptance is the milestone's final gate (B1): a merged PR is the
|
||||||
|
# durable record, so completion returns to planning or to idle rather
|
||||||
|
# than passing through a separate FINISHED state.
|
||||||
|
(PR_OPEN, MILESTONE_PLANNING),
|
||||||
|
(PR_OPEN, IDLE),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -59,6 +69,42 @@ class AskHuman:
|
|||||||
question: str
|
question: str
|
||||||
|
|
||||||
|
|
||||||
|
class MilestoneExecutor(Protocol):
|
||||||
|
"""What the orchestrator needs from an executor loop; the concrete one
|
||||||
|
lives in nisse_dev.executor (imported there, not here, to keep the
|
||||||
|
dependency pointing one way)."""
|
||||||
|
|
||||||
|
def run(self, milestone: int) -> Any: ...
|
||||||
|
|
||||||
|
|
||||||
|
class MilestonePublisher(Protocol):
|
||||||
|
"""What the orchestrator needs to publish a milestone; the concrete one
|
||||||
|
lives in nisse_dev.publish."""
|
||||||
|
|
||||||
|
def reconcilers(self) -> Mapping[str, Reconciler]: ...
|
||||||
|
|
||||||
|
def publish(
|
||||||
|
self, *, task_id: str, milestone: int, project: str, plan: str
|
||||||
|
) -> Any: ...
|
||||||
|
|
||||||
|
def push_followup(
|
||||||
|
self, *, task_id: str, milestone: int, message: str
|
||||||
|
) -> str | None: ...
|
||||||
|
|
||||||
|
def comment(self, index: int, body: str) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
|
class PlanBoard(Protocol):
|
||||||
|
"""The Gitea side of plan gating; the concrete one is in
|
||||||
|
nisse_dev.approvals."""
|
||||||
|
|
||||||
|
def open_project_issue(self, project: str, task_id: str) -> int: ...
|
||||||
|
|
||||||
|
def post_plan(self, issue: int, gate: str, rev: int, plan: str) -> Any: ...
|
||||||
|
|
||||||
|
def echo(self, issue: int, text: str) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
class PlanningAgent(Protocol):
|
class PlanningAgent(Protocol):
|
||||||
def plan(
|
def plan(
|
||||||
self,
|
self,
|
||||||
@@ -91,10 +137,17 @@ def _fresh_data() -> dict[str, Any]:
|
|||||||
return {
|
return {
|
||||||
"revs": {},
|
"revs": {},
|
||||||
"gate": None,
|
"gate": None,
|
||||||
"question": None,
|
# What an operator reply in AWAIT_INPUT should resume. Every path into
|
||||||
|
# AWAIT_INPUT must set this, or the reply has nowhere to go.
|
||||||
|
"pending": None,
|
||||||
"milestone": 1,
|
"milestone": 1,
|
||||||
"task_id": None,
|
"task_id": None,
|
||||||
"roles": None, # A11 mapping, pinned at task start
|
"roles": None, # A11 mapping, pinned at task start
|
||||||
|
"pr": None, # the open PR for the current milestone, once published
|
||||||
|
# Pointer to the Gitea plan record (Part H): the state store holds
|
||||||
|
# only this, so gate state can be re-derived from Gitea after a
|
||||||
|
# state-volume loss.
|
||||||
|
"board": None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -106,18 +159,37 @@ class Orchestrator:
|
|||||||
agent: PlanningAgent,
|
agent: PlanningAgent,
|
||||||
operators: Iterable[str],
|
operators: Iterable[str],
|
||||||
roles: RoleMap | None = None,
|
roles: RoleMap | None = None,
|
||||||
|
executor: "MilestoneExecutor | None" = None,
|
||||||
|
publisher: "MilestonePublisher | None" = None,
|
||||||
|
gitea_operators: Iterable[str] = (),
|
||||||
|
board: "PlanBoard | None" = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._store = store
|
self._store = store
|
||||||
self._transport = transport
|
self._transport = transport
|
||||||
self._agent = agent
|
self._agent = agent
|
||||||
self._roles = roles
|
self._roles = roles
|
||||||
|
self._executor = executor
|
||||||
|
self._publisher = publisher
|
||||||
self._operators = set(operators)
|
self._operators = set(operators)
|
||||||
|
# Per-channel allowlists (A4): a Matrix MXID and a Gitea login
|
||||||
|
# are different names for the same person, and neither implies
|
||||||
|
# the other.
|
||||||
|
self._gitea_operators = set(gitea_operators)
|
||||||
|
self._board = board
|
||||||
|
# Transport and the comment poller deliver on separate threads;
|
||||||
|
# state transitions must not interleave.
|
||||||
|
self._lock = threading.RLock()
|
||||||
self._data = _fresh_data()
|
self._data = _fresh_data()
|
||||||
self._project: str | None = None
|
self._project: str | None = None
|
||||||
transport.on_message(self._handle)
|
transport.on_message(self._handle)
|
||||||
|
|
||||||
def start(self, reconcilers: Mapping[str, Reconciler] | None = None) -> None:
|
def start(self, reconcilers: Mapping[str, Reconciler] | None = None) -> None:
|
||||||
report = reconcile_on_boot(self._store, reconcilers or {})
|
# The publisher owns the external-truth query for its own intents, so
|
||||||
|
# a configured publisher always contributes its reconciler (A3).
|
||||||
|
registered: dict[str, Reconciler] = dict(reconcilers or {})
|
||||||
|
if self._publisher is not None:
|
||||||
|
registered = {**self._publisher.reconcilers(), **registered}
|
||||||
|
report = reconcile_on_boot(self._store, registered)
|
||||||
state = self._store.get_state()
|
state = self._store.get_state()
|
||||||
if state is None:
|
if state is None:
|
||||||
self._transition(IDLE, cause="boot: fresh state")
|
self._transition(IDLE, cause="boot: fresh state")
|
||||||
@@ -128,11 +200,14 @@ class Orchestrator:
|
|||||||
self._store.oplog.task_id = self._data.get("task_id")
|
self._store.oplog.task_id = self._data.get("task_id")
|
||||||
gate = self._data.get("gate")
|
gate = self._data.get("gate")
|
||||||
if report.action == "await_input":
|
if report.action == "await_input":
|
||||||
self._transition(AWAIT_INPUT, cause="boot: unresolved intents")
|
self._await_input(
|
||||||
self._transport.ask_question(
|
cause="boot: unresolved intents",
|
||||||
|
prompt=(
|
||||||
"restart found unresolved work: "
|
"restart found unresolved work: "
|
||||||
+ "; ".join(report.notes)
|
+ "; ".join(report.notes)
|
||||||
+ " — how should I proceed?"
|
+ " — how should I proceed?"
|
||||||
|
),
|
||||||
|
pending={"kind": "boot"},
|
||||||
)
|
)
|
||||||
elif state.state == IDLE or (
|
elif state.state == IDLE or (
|
||||||
state.state in (AWAIT_ARCH_SIGNOFF, AWAIT_MILESTONE_SIGNOFF) and gate
|
state.state in (AWAIT_ARCH_SIGNOFF, AWAIT_MILESTONE_SIGNOFF) and gate
|
||||||
@@ -147,12 +222,17 @@ class Orchestrator:
|
|||||||
else:
|
else:
|
||||||
# Interrupted mid-turn (planning, executing, unanswered question):
|
# Interrupted mid-turn (planning, executing, unanswered question):
|
||||||
# never blind-resume (A3.6).
|
# never blind-resume (A3.6).
|
||||||
self._transition(AWAIT_INPUT, cause=f"boot: restarted in {state.state}")
|
self._await_input(
|
||||||
self._transport.ask_question(
|
cause=f"boot: restarted in {state.state}",
|
||||||
f"restarted mid-{state.state}; continue or re-plan?"
|
prompt=f"restarted mid-{state.state}; continue or re-plan?",
|
||||||
|
pending={"kind": "boot"},
|
||||||
)
|
)
|
||||||
|
|
||||||
def _handle(self, msg: InboundMessage) -> None:
|
def _handle(self, msg: InboundMessage) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._handle_locked(msg)
|
||||||
|
|
||||||
|
def _handle_locked(self, msg: InboundMessage) -> None:
|
||||||
if msg.sender not in self._operators:
|
if msg.sender not in self._operators:
|
||||||
return
|
return
|
||||||
# Every operator message enters the canonical transcript, so planning
|
# Every operator message enters the canonical transcript, so planning
|
||||||
@@ -173,6 +253,8 @@ class Orchestrator:
|
|||||||
self._transport.send_message("no active project; say 'use <project>'")
|
self._transport.send_message("no active project; say 'use <project>'")
|
||||||
elif current in (AWAIT_ARCH_SIGNOFF, AWAIT_MILESTONE_SIGNOFF):
|
elif current in (AWAIT_ARCH_SIGNOFF, AWAIT_MILESTONE_SIGNOFF):
|
||||||
self._handle_gate_reply(msg, text)
|
self._handle_gate_reply(msg, text)
|
||||||
|
elif current == PR_OPEN:
|
||||||
|
self._handle_pr_open(text)
|
||||||
elif current == AWAIT_INPUT:
|
elif current == AWAIT_INPUT:
|
||||||
self._handle_answer(text)
|
self._handle_answer(text)
|
||||||
else:
|
else:
|
||||||
@@ -195,6 +277,19 @@ class Orchestrator:
|
|||||||
self._data = {**_fresh_data(), "task_id": task_id, "roles": pinned}
|
self._data = {**_fresh_data(), "task_id": task_id, "roles": pinned}
|
||||||
self._project = project
|
self._project = project
|
||||||
self._store.oplog.task_id = task_id
|
self._store.oplog.task_id = task_id
|
||||||
|
if self._board is not None:
|
||||||
|
try:
|
||||||
|
self._data["board"] = {
|
||||||
|
"issue": self._board.open_project_issue(project, task_id),
|
||||||
|
"last_comment": 0,
|
||||||
|
}
|
||||||
|
except GiteaError as exc:
|
||||||
|
# Gitea being down must not block planning; Matrix remains a
|
||||||
|
# complete approval channel on its own.
|
||||||
|
self._transport.send_message(
|
||||||
|
f"note: could not open the Gitea plan issue ({exc}); "
|
||||||
|
"approvals will be Matrix-only for this task"
|
||||||
|
)
|
||||||
self._transition(PLANNING_ARCH, cause=f"operator command: use {project}")
|
self._transition(PLANNING_ARCH, cause=f"operator command: use {project}")
|
||||||
self._run_planning_turn("arch")
|
self._run_planning_turn("arch")
|
||||||
|
|
||||||
@@ -207,17 +302,18 @@ class Orchestrator:
|
|||||||
gate_name, self._project, feedback=feedback, answer=answer
|
gate_name, self._project, feedback=feedback, answer=answer
|
||||||
)
|
)
|
||||||
except PlanningError as exc:
|
except PlanningError as exc:
|
||||||
self._transition(AWAIT_INPUT, cause=f"planning failed: {exc}")
|
self._await_input(
|
||||||
self._transport.ask_question(
|
cause=f"planning failed: {exc}",
|
||||||
f"planning for {gate_name} failed ({exc}); retry or re-plan?"
|
prompt=f"planning for {gate_name} failed ({exc}); how should I proceed?",
|
||||||
|
pending={"kind": "planning_failed", "gate": gate_name},
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
if isinstance(action, AskHuman):
|
if isinstance(action, AskHuman):
|
||||||
self._data["question"] = {"text": action.question, "gate": gate_name}
|
self._await_input(
|
||||||
self._transition(
|
cause=f"agent question during {gate_name} planning",
|
||||||
AWAIT_INPUT, cause=f"agent question during {gate_name} planning"
|
prompt=action.question,
|
||||||
|
pending={"kind": "question", "gate": gate_name},
|
||||||
)
|
)
|
||||||
self._transport.ask_question(action.question)
|
|
||||||
return
|
return
|
||||||
rev = self._data["revs"].get(gate_name, 0) + 1
|
rev = self._data["revs"].get(gate_name, 0) + 1
|
||||||
self._data["revs"][gate_name] = rev
|
self._data["revs"][gate_name] = rev
|
||||||
@@ -242,6 +338,7 @@ class Orchestrator:
|
|||||||
f"{action.content}\n\nreply 'approve' to approve rev-{rev}; "
|
f"{action.content}\n\nreply 'approve' to approve rev-{rev}; "
|
||||||
"any other reply is rejection feedback"
|
"any other reply is rejection feedback"
|
||||||
)
|
)
|
||||||
|
self._post_plan_to_board(gate_name, rev, action.content)
|
||||||
self._store.oplog.emit(
|
self._store.oplog.emit(
|
||||||
"gate.open",
|
"gate.open",
|
||||||
cause=f"plan submitted (rev-{rev})",
|
cause=f"plan submitted (rev-{rev})",
|
||||||
@@ -251,6 +348,12 @@ class Orchestrator:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def _handle_gate_reply(self, msg: InboundMessage, text: str) -> None:
|
def _handle_gate_reply(self, msg: InboundMessage, text: str) -> None:
|
||||||
|
self._apply_gate_reply(sender=msg.sender, text=text, channel="transport")
|
||||||
|
|
||||||
|
def _apply_gate_reply(self, *, sender: str, text: str, channel: str) -> None:
|
||||||
|
"""Shared by every approval source (A4). The allowlist is per channel:
|
||||||
|
a Matrix MXID and a Gitea login are different names, and holding one
|
||||||
|
says nothing about holding the other."""
|
||||||
gate_data = self._data["gate"]
|
gate_data = self._data["gate"]
|
||||||
name, rev = gate_data["name"], gate_data["rev"]
|
name, rev = gate_data["name"], gate_data["rev"]
|
||||||
milestone = self._data["milestone"]
|
milestone = self._data["milestone"]
|
||||||
@@ -259,16 +362,15 @@ class Orchestrator:
|
|||||||
self._transition(
|
self._transition(
|
||||||
EXECUTING, cause=f"operator command: start milestone {milestone}"
|
EXECUTING, cause=f"operator command: start milestone {milestone}"
|
||||||
)
|
)
|
||||||
self._transport.send_message(f"[stub] executing milestone {milestone}")
|
self._execute_milestone(milestone)
|
||||||
else:
|
else:
|
||||||
self._transport.send_message(
|
self._transport.send_message(
|
||||||
f"plan approved; say 'start milestone {milestone}' to begin"
|
f"plan approved; say 'start milestone {milestone}' to begin"
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
revision = PlanRevision(gate=name, rev=rev, content=gate_data["plan"])
|
revision = PlanRevision(gate=name, rev=rev, content=gate_data["plan"])
|
||||||
result = Gate(revision, self._operators).evaluate(
|
allowlist = self._operators if channel == "transport" else self._gitea_operators
|
||||||
msg.sender, text, channel="transport"
|
result = Gate(revision, allowlist).evaluate(sender, text, channel=channel)
|
||||||
)
|
|
||||||
if isinstance(result, Ignored):
|
if isinstance(result, Ignored):
|
||||||
if result.notice:
|
if result.notice:
|
||||||
self._transport.send_message(result.notice)
|
self._transport.send_message(result.notice)
|
||||||
@@ -281,12 +383,19 @@ class Orchestrator:
|
|||||||
rev=f"rev-{rev}",
|
rev=f"rev-{rev}",
|
||||||
channel=result.channel,
|
channel=result.channel,
|
||||||
)
|
)
|
||||||
|
self._echo_outcome(
|
||||||
|
channel,
|
||||||
|
f"{name} rev-{rev} rejected by {result.approver}: {result.feedback}",
|
||||||
|
)
|
||||||
self._data["gate"] = None
|
self._data["gate"] = None
|
||||||
back = PLANNING_ARCH if name == "arch" else MILESTONE_PLANNING
|
back = PLANNING_ARCH if name == "arch" else MILESTONE_PLANNING
|
||||||
self._transition(back, cause=f"plan rejected ({name} rev-{rev})")
|
self._transition(back, cause=f"plan rejected ({name} rev-{rev})")
|
||||||
self._run_planning_turn(name, feedback=result.feedback)
|
self._run_planning_turn(name, feedback=result.feedback)
|
||||||
return
|
return
|
||||||
assert isinstance(result, Approved)
|
assert isinstance(result, Approved)
|
||||||
|
self._echo_outcome(
|
||||||
|
channel, f"{name} rev-{rev} approved by {result.approver} via {channel}"
|
||||||
|
)
|
||||||
self._store.oplog.emit(
|
self._store.oplog.emit(
|
||||||
"gate.approve",
|
"gate.approve",
|
||||||
cause=f"approved by {result.approver}",
|
cause=f"approved by {result.approver}",
|
||||||
@@ -313,18 +422,227 @@ class Orchestrator:
|
|||||||
f"milestone plan approved; say 'start milestone {milestone}' to begin"
|
f"milestone plan approved; say 'start milestone {milestone}' to begin"
|
||||||
)
|
)
|
||||||
|
|
||||||
def _handle_answer(self, text: str) -> None:
|
def _execute_milestone(self, milestone: int) -> None:
|
||||||
question = self._data.get("question")
|
if self._executor is None:
|
||||||
if question is None:
|
self._transport.send_message(f"[stub] executing milestone {milestone}")
|
||||||
self._transport.send_message(
|
return
|
||||||
"awaiting operator resolution; no pending question to answer"
|
self._transport.send_message(f"executing milestone {milestone}")
|
||||||
|
outcome = self._executor.run(milestone)
|
||||||
|
if outcome.status == "completed":
|
||||||
|
self._transport.send_message(f"milestone {milestone} done: {outcome.note}")
|
||||||
|
self._publish_milestone(milestone)
|
||||||
|
return
|
||||||
|
self._await_input(
|
||||||
|
cause=f"milestone {milestone} {outcome.status}",
|
||||||
|
prompt=f"milestone {milestone} stopped ({outcome.status}): {outcome.note}",
|
||||||
|
pending={"kind": "milestone_stopped", "milestone": milestone},
|
||||||
|
)
|
||||||
|
|
||||||
|
def _echo_outcome(self, deciding_channel: str, text: str) -> None:
|
||||||
|
"""A gate satisfied on one channel is echoed to the other, so neither
|
||||||
|
record silently disagrees about what was approved (A4)."""
|
||||||
|
board_state = self._data.get("board")
|
||||||
|
if deciding_channel != "gitea" and self._board is not None and board_state:
|
||||||
|
self._board.echo(board_state["issue"], text)
|
||||||
|
elif deciding_channel == "gitea":
|
||||||
|
self._transport.send_message(text)
|
||||||
|
|
||||||
|
def _post_plan_to_board(self, gate_name: str, rev: int, plan: str) -> None:
|
||||||
|
board_state = self._data.get("board")
|
||||||
|
if self._board is None or not board_state:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
post = self._board.post_plan(board_state["issue"], gate_name, rev, plan)
|
||||||
|
except GiteaError:
|
||||||
|
# Matrix already carries the plan; a Gitea outage degrades the
|
||||||
|
# durability of the record, not the ability to approve.
|
||||||
|
return
|
||||||
|
board_state["comment_id"] = post.comment_id
|
||||||
|
|
||||||
|
def handle_plan_comment(self, comment: Any) -> None:
|
||||||
|
"""A comment on the project issue may satisfy an open plan gate (A4).
|
||||||
|
|
||||||
|
Approvals bind to a revision here exactly as over transport, and the
|
||||||
|
first valid approval on either channel wins.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
self._handle_plan_comment_locked(comment)
|
||||||
|
|
||||||
|
def _handle_plan_comment_locked(self, comment: Any) -> None:
|
||||||
|
if comment.author not in self._gitea_operators:
|
||||||
|
return
|
||||||
|
state = self._store.get_state()
|
||||||
|
if state is None or state.state not in (
|
||||||
|
AWAIT_ARCH_SIGNOFF,
|
||||||
|
AWAIT_MILESTONE_SIGNOFF,
|
||||||
|
):
|
||||||
|
return
|
||||||
|
self._store.append_event(
|
||||||
|
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
||||||
|
)
|
||||||
|
self._apply_gate_reply(
|
||||||
|
sender=comment.author, text=comment.body.strip(), channel="gitea"
|
||||||
|
)
|
||||||
|
|
||||||
|
def handle_review_comment(self, comment: Any) -> None:
|
||||||
|
"""React to a PR review comment (B2). Called from the poller thread.
|
||||||
|
|
||||||
|
Comment authors are checked against the Gitea operator allowlist:
|
||||||
|
a PR comment is an instruction channel into an agent that writes
|
||||||
|
code, so it is operator-scoped like any other (A4, and the same
|
||||||
|
reasoning C4 applies to tickets).
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
self._handle_review_comment_locked(comment)
|
||||||
|
|
||||||
|
def _handle_review_comment_locked(self, comment: Any) -> None:
|
||||||
|
if comment.author not in self._gitea_operators:
|
||||||
|
self._store.oplog.emit(
|
||||||
|
"gitea.comment_ignored",
|
||||||
|
cause=f"author {comment.author!r} not an operator",
|
||||||
|
comment_id=comment.id,
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
self._data["question"] = None
|
state = self._store.get_state()
|
||||||
gate_name = question["gate"]
|
if state is None or state.state != PR_OPEN:
|
||||||
|
self._store.oplog.emit(
|
||||||
|
"gitea.comment_ignored",
|
||||||
|
cause=f"not in PR_OPEN (state={state.state if state else None})",
|
||||||
|
comment_id=comment.id,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
if self._executor is None or self._publisher is None:
|
||||||
|
return
|
||||||
|
|
||||||
|
pr = self._data.get("pr") or {}
|
||||||
|
milestone = int(pr.get("milestone", self._data.get("milestone", 1)))
|
||||||
|
# The comment enters the transcript, so the executor sees review
|
||||||
|
# feedback as conversation rather than as an out-of-band instruction.
|
||||||
|
self._store.append_event(
|
||||||
|
UserMessage(text=comment.body, source=f"gitea:comment:{comment.id}")
|
||||||
|
)
|
||||||
|
outcome = self._executor.run(milestone)
|
||||||
|
if outcome.status != "completed":
|
||||||
|
self._await_input(
|
||||||
|
cause=f"review response {outcome.status}",
|
||||||
|
prompt=(
|
||||||
|
f"could not address the review comment ({outcome.status}): "
|
||||||
|
f"{outcome.note}"
|
||||||
|
),
|
||||||
|
pending={"kind": "milestone_stopped", "milestone": milestone},
|
||||||
|
)
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
sha = self._publisher.push_followup(
|
||||||
|
task_id=str(self._data.get("task_id")),
|
||||||
|
milestone=milestone,
|
||||||
|
message=f"Address review comment {comment.id}",
|
||||||
|
)
|
||||||
|
except (GiteaError, GitError) as exc:
|
||||||
|
self._transport.send_message(f"could not push the fix: {exc}")
|
||||||
|
return
|
||||||
|
note = (
|
||||||
|
f"pushed {sha[:8]} addressing the review comment"
|
||||||
|
if sha
|
||||||
|
else "no code change was needed for that comment"
|
||||||
|
)
|
||||||
|
self._transport.send_message(note)
|
||||||
|
if pr.get("number"):
|
||||||
|
self._publisher.comment(int(pr["number"]), note)
|
||||||
|
|
||||||
|
def _publish_milestone(self, milestone: int) -> None:
|
||||||
|
if self._publisher is None:
|
||||||
|
return
|
||||||
|
gate = self._data.get("gate") or {}
|
||||||
|
try:
|
||||||
|
result = self._publisher.publish(
|
||||||
|
task_id=str(self._data.get("task_id")),
|
||||||
|
milestone=milestone,
|
||||||
|
project=str(self._project),
|
||||||
|
plan=str(gate.get("plan", "")),
|
||||||
|
)
|
||||||
|
except (GiteaError, GitError) as exc:
|
||||||
|
# The intent log already records what was in flight, so boot
|
||||||
|
# reconciliation can settle it against Gitea. Gate rather than
|
||||||
|
# retry blindly into a possible duplicate.
|
||||||
|
self._await_input(
|
||||||
|
cause=f"publishing milestone {milestone} failed: {exc}",
|
||||||
|
prompt=f"could not publish milestone {milestone}: {exc}",
|
||||||
|
pending={"kind": "milestone_stopped", "milestone": milestone},
|
||||||
|
)
|
||||||
|
return
|
||||||
|
if result.pull_request is None:
|
||||||
|
self._transport.send_message(result.note)
|
||||||
|
return
|
||||||
|
self._data["pr"] = {
|
||||||
|
"number": result.pull_request.number,
|
||||||
|
"url": result.pull_request.url,
|
||||||
|
"milestone": milestone,
|
||||||
|
}
|
||||||
|
self._transition(PR_OPEN, cause=f"milestone {milestone} published")
|
||||||
|
self._transport.send_message(
|
||||||
|
f"{result.note} — review it, then say 'next milestone' or "
|
||||||
|
"'project complete'"
|
||||||
|
)
|
||||||
|
|
||||||
|
def _handle_pr_open(self, text: str) -> None:
|
||||||
|
if text == "next milestone":
|
||||||
|
self._data["milestone"] = int(self._data.get("milestone", 1)) + 1
|
||||||
|
self._data["gate"] = None
|
||||||
|
self._data["pr"] = None
|
||||||
|
self._transition(MILESTONE_PLANNING, cause="operator: next milestone")
|
||||||
|
self._run_planning_turn(f"milestone-{self._data['milestone']}")
|
||||||
|
elif text == "project complete":
|
||||||
|
self._data = {**_fresh_data()}
|
||||||
|
self._project = None
|
||||||
|
self._store.oplog.task_id = None
|
||||||
|
self._transition(IDLE, cause="operator: project complete")
|
||||||
|
self._transport.send_message(
|
||||||
|
"project closed; say 'use <project>' to start another"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
pr = self._data.get("pr") or {}
|
||||||
|
self._transport.send_message(
|
||||||
|
f"PR #{pr.get('number')} is open; say 'next milestone' or "
|
||||||
|
"'project complete'"
|
||||||
|
)
|
||||||
|
|
||||||
|
def _await_input(self, *, cause: str, prompt: str, pending: dict[str, Any]) -> None:
|
||||||
|
self._data["pending"] = pending
|
||||||
|
self._transition(AWAIT_INPUT, cause=cause)
|
||||||
|
self._transport.ask_question(prompt)
|
||||||
|
|
||||||
|
def _handle_answer(self, text: str) -> None:
|
||||||
|
pending = self._data.get("pending")
|
||||||
|
if pending is None:
|
||||||
|
# Reached only if state was forced externally; say what works
|
||||||
|
# rather than dead-ending the operator.
|
||||||
|
self._transport.send_message(
|
||||||
|
"waiting for direction, but I have no pending item to resume; "
|
||||||
|
"try 'status', or 'use <project>' to start over"
|
||||||
|
)
|
||||||
|
return
|
||||||
|
self._data["pending"] = None
|
||||||
|
match pending.get("kind"):
|
||||||
|
case "question" | "planning_failed":
|
||||||
|
gate_name = pending["gate"]
|
||||||
back = PLANNING_ARCH if gate_name == "arch" else MILESTONE_PLANNING
|
back = PLANNING_ARCH if gate_name == "arch" else MILESTONE_PLANNING
|
||||||
self._transition(back, cause="operator answered question")
|
self._transition(back, cause="operator answered; resuming planning")
|
||||||
self._run_planning_turn(gate_name, answer=text)
|
self._run_planning_turn(gate_name, answer=text)
|
||||||
|
case "milestone_stopped":
|
||||||
|
milestone = pending["milestone"]
|
||||||
|
self._transition(
|
||||||
|
AWAIT_MILESTONE_SIGNOFF,
|
||||||
|
cause="operator answered; plan still approved",
|
||||||
|
)
|
||||||
|
self._transport.send_message(
|
||||||
|
f"noted. the milestone {milestone} plan is still approved — "
|
||||||
|
f"say 'start milestone {milestone}' to run it again"
|
||||||
|
)
|
||||||
|
case _:
|
||||||
|
self._transport.send_message(
|
||||||
|
"noted. say 'status' for where things stand"
|
||||||
|
)
|
||||||
|
|
||||||
def _transition(self, new_state: str, *, cause: str) -> None:
|
def _transition(self, new_state: str, *, cause: str) -> None:
|
||||||
state = self._store.get_state()
|
state = self._store.get_state()
|
||||||
|
|||||||
@@ -0,0 +1,363 @@
|
|||||||
|
"""Delegation harness (spec A13).
|
||||||
|
|
||||||
|
Escalation is delegation, not consultation: a frontier model receives a scoped
|
||||||
|
sub-task, works autonomously with full tool access, and hands back. The local
|
||||||
|
model does not interpret or re-derive its reasoning — the handback note plus
|
||||||
|
the resulting file state is the entire interface back.
|
||||||
|
|
||||||
|
Three properties do the work here:
|
||||||
|
|
||||||
|
**Isolation.** The delegation runs on its own branch, taken at start. Abort
|
||||||
|
discards the branch, which gives a delegation the same atomicity that boot
|
||||||
|
reconciliation gives an interrupted state transition.
|
||||||
|
|
||||||
|
**Caps that are hard.** Turns, tokens, and wall clock. A breach is a defined
|
||||||
|
failure return, not a nudge: the local loop gates to the human rather than
|
||||||
|
resuming on top of a half-edited tree.
|
||||||
|
|
||||||
|
**Checkpointing as it happens.** Every tool call, result, and assistant turn
|
||||||
|
is written to the canonical log when it occurs, never batched at handback.
|
||||||
|
That is what makes a delegation resumable after a crash (A3).
|
||||||
|
|
||||||
|
Scope is bounded by construction: a delegation is handed a sub-task the
|
||||||
|
approved plan already contains. It is smart, not authorised to widen the work.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import time
|
||||||
|
from collections.abc import Callable
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from nisse_dev.canonical import (
|
||||||
|
AssistantText,
|
||||||
|
DelegationCompleted,
|
||||||
|
DelegationStarted,
|
||||||
|
ToolCall,
|
||||||
|
ToolResult,
|
||||||
|
)
|
||||||
|
from nisse_dev.git import GitError, GitRepo
|
||||||
|
from nisse_dev.llm import LLMClient, LLMError
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
|
||||||
|
|
||||||
|
DELEGATION_INTENT = "delegation"
|
||||||
|
|
||||||
|
SYSTEM_PROMPT = """You are a delegate working on one scoped sub-task for
|
||||||
|
nisse-dev. You have full tool access and are expected to finish the sub-task
|
||||||
|
yourself, including running the tests.
|
||||||
|
|
||||||
|
Stay inside the sub-task you were given. It comes from a plan a human already
|
||||||
|
approved; widening it is not yours to do. If the sub-task turns out to be
|
||||||
|
wrong or impossible, say so in the handback rather than substituting your own.
|
||||||
|
|
||||||
|
When you are done, call hand_back exactly once. Write it for a materially
|
||||||
|
weaker model that cannot see your reasoning and will act on your note alone:
|
||||||
|
|
||||||
|
- what changed
|
||||||
|
- what is now true that was not before
|
||||||
|
- what to do next
|
||||||
|
|
||||||
|
Do not call hand_back until the work is actually done."""
|
||||||
|
|
||||||
|
HAND_BACK_TOOL: dict[str, Any] = {
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "hand_back",
|
||||||
|
"description": (
|
||||||
|
"End the delegation and report. Call exactly once, when the "
|
||||||
|
"sub-task is complete or you have determined it cannot be."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"changed": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "What you changed, concretely.",
|
||||||
|
},
|
||||||
|
"now_true": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "What is now true that was not before.",
|
||||||
|
},
|
||||||
|
"next": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "What the caller should do next.",
|
||||||
|
},
|
||||||
|
"succeeded": {
|
||||||
|
"type": "boolean",
|
||||||
|
"description": "Whether the sub-task was completed.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": ["changed", "now_true", "next", "succeeded"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
DELEGATE_TOOLS = [*EXECUTOR_TOOLS, HAND_BACK_TOOL]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Caps:
|
||||||
|
turns: int = 30
|
||||||
|
tokens: int = 200_000
|
||||||
|
wall_clock_s: float = 900.0
|
||||||
|
|
||||||
|
def as_payload(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"turns": self.turns,
|
||||||
|
"tokens": self.tokens,
|
||||||
|
"wall_clock_s": self.wall_clock_s,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Handback:
|
||||||
|
"""The entire interface back to the local loop, alongside file state."""
|
||||||
|
|
||||||
|
changed: str
|
||||||
|
now_true: str
|
||||||
|
next: str
|
||||||
|
succeeded: bool
|
||||||
|
|
||||||
|
def render(self) -> str:
|
||||||
|
return (
|
||||||
|
f"What changed: {self.changed}\n"
|
||||||
|
f"What is now true: {self.now_true}\n"
|
||||||
|
f"What to do next: {self.next}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class DelegationResult:
|
||||||
|
status: Literal["handback", "capped", "failed"]
|
||||||
|
note: str
|
||||||
|
handback: Handback | None = None
|
||||||
|
branch: str | None = None
|
||||||
|
artifacts: list[str] = field(default_factory=list)
|
||||||
|
tokens: int = 0
|
||||||
|
duration_s: float = 0.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def usable(self) -> bool:
|
||||||
|
"""Only a successful handback may be built upon. A cap breach or a
|
||||||
|
failure leaves a half-edited tree, which A13 says must gate rather
|
||||||
|
than be resumed."""
|
||||||
|
return self.status == "handback" and self.handback is not None
|
||||||
|
|
||||||
|
|
||||||
|
class Delegation:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
client: LLMClient,
|
||||||
|
store: StateStore,
|
||||||
|
workspace: Workspace,
|
||||||
|
git_repo: GitRepo,
|
||||||
|
*,
|
||||||
|
model: str,
|
||||||
|
caps: Caps | None = None,
|
||||||
|
now: Callable[[], float] = time.monotonic,
|
||||||
|
) -> None:
|
||||||
|
self._client = client
|
||||||
|
self._store = store
|
||||||
|
self._workspace = workspace
|
||||||
|
self._git = git_repo
|
||||||
|
self._model = model
|
||||||
|
self._caps = caps or Caps()
|
||||||
|
self._now = now
|
||||||
|
|
||||||
|
def run(
|
||||||
|
self, *, brief: str, branch: str, role: str = "delegate"
|
||||||
|
) -> DelegationResult:
|
||||||
|
started_at = self._now()
|
||||||
|
base_branch = self._git.current_branch()
|
||||||
|
try:
|
||||||
|
# Commit whatever the caller had in flight onto their own branch
|
||||||
|
# first. Otherwise it would ride along onto the delegation branch
|
||||||
|
# and be destroyed by an abort — discarding work nobody delegated.
|
||||||
|
self._git.commit_all("Checkpoint before delegating")
|
||||||
|
self._git.checkout_new_branch(branch)
|
||||||
|
except GitError as exc:
|
||||||
|
return DelegationResult("failed", f"could not branch for delegation: {exc}")
|
||||||
|
|
||||||
|
self._store.append_event(
|
||||||
|
DelegationStarted(
|
||||||
|
brief=brief,
|
||||||
|
role=role,
|
||||||
|
model=self._model,
|
||||||
|
caps=self._caps.as_payload(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
result = self._loop(brief, branch, started_at)
|
||||||
|
result.duration_s = self._now() - started_at
|
||||||
|
self._store.append_event(
|
||||||
|
DelegationCompleted(
|
||||||
|
handback=result.handback.render() if result.handback else result.note,
|
||||||
|
artifacts=result.artifacts,
|
||||||
|
tokens=result.tokens,
|
||||||
|
duration_s=result.duration_s,
|
||||||
|
exit_status=result.status,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if not result.usable:
|
||||||
|
# Abort discards the branch: the caller must never resume on top
|
||||||
|
# of a partially-edited tree (A13).
|
||||||
|
self._abandon(branch, base_branch)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _loop(self, brief: str, branch: str, started_at: float) -> DelegationResult:
|
||||||
|
system = (
|
||||||
|
f"{SYSTEM_PROMPT}\n\n{self._workspace.describe()}"
|
||||||
|
f"\n\nYour sub-task:\n{brief}"
|
||||||
|
)
|
||||||
|
touched: set[str] = set()
|
||||||
|
did_work = False
|
||||||
|
|
||||||
|
for turn in range(self._caps.turns):
|
||||||
|
elapsed = self._now() - started_at
|
||||||
|
if elapsed >= self._caps.wall_clock_s:
|
||||||
|
return DelegationResult(
|
||||||
|
"capped",
|
||||||
|
f"wall-clock cap of {self._caps.wall_clock_s}s reached "
|
||||||
|
f"after {turn} turns",
|
||||||
|
branch=branch,
|
||||||
|
artifacts=sorted(touched),
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
produced = self._client.complete(
|
||||||
|
self._store.read_events(),
|
||||||
|
tools=DELEGATE_TOOLS,
|
||||||
|
model=self._model,
|
||||||
|
system=system,
|
||||||
|
)
|
||||||
|
except LLMError as exc:
|
||||||
|
return DelegationResult(
|
||||||
|
"failed", f"delegate call failed: {exc}", branch=branch
|
||||||
|
)
|
||||||
|
|
||||||
|
calls: list[ToolCall] = []
|
||||||
|
for event in produced:
|
||||||
|
# Checkpoint as it arrives, never batched at handback (A13):
|
||||||
|
# this is what makes the delegation resumable after a crash.
|
||||||
|
self._store.append_event(event)
|
||||||
|
if isinstance(event, ToolCall):
|
||||||
|
calls.append(event)
|
||||||
|
|
||||||
|
if not calls:
|
||||||
|
# A delegate that stops without handing back has not reported,
|
||||||
|
# and its tree state is unknown. Treat it as a failure rather
|
||||||
|
# than inventing a handback on its behalf.
|
||||||
|
if any(isinstance(e, AssistantText) for e in produced):
|
||||||
|
continue
|
||||||
|
return DelegationResult(
|
||||||
|
"failed",
|
||||||
|
"delegate produced neither a tool call nor text",
|
||||||
|
branch=branch,
|
||||||
|
artifacts=sorted(touched),
|
||||||
|
)
|
||||||
|
|
||||||
|
for call in calls:
|
||||||
|
if call.tool == "hand_back":
|
||||||
|
handback = _parse_handback(call.arguments)
|
||||||
|
if handback is not None and handback.succeeded and not did_work:
|
||||||
|
# Success claimed with nothing changed. The handback
|
||||||
|
# and the file state are the whole interface back, so
|
||||||
|
# an unbacked claim would hand the caller a note about
|
||||||
|
# work that does not exist.
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=call.call_id,
|
||||||
|
result=(
|
||||||
|
"you have not changed anything yet, so the "
|
||||||
|
"sub-task cannot be complete; do the work, "
|
||||||
|
"or hand back with succeeded=false and say "
|
||||||
|
"why"
|
||||||
|
),
|
||||||
|
success=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if handback is None:
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=call.call_id,
|
||||||
|
result=(
|
||||||
|
"hand_back needs changed, now_true, next and "
|
||||||
|
"succeeded; the delegation is still open"
|
||||||
|
),
|
||||||
|
success=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=call.call_id,
|
||||||
|
result="handback accepted; delegation closed",
|
||||||
|
success=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self._commit_turn(turn)
|
||||||
|
return DelegationResult(
|
||||||
|
"handback" if handback.succeeded else "failed",
|
||||||
|
handback.next if handback.succeeded else handback.changed,
|
||||||
|
handback=handback if handback.succeeded else None,
|
||||||
|
branch=branch,
|
||||||
|
artifacts=sorted(touched),
|
||||||
|
)
|
||||||
|
|
||||||
|
outcome = self._workspace.dispatch(call.tool, call.arguments)
|
||||||
|
if call.tool in MUTATING_TOOLS and outcome.success:
|
||||||
|
did_work = True
|
||||||
|
path = call.arguments.get("path")
|
||||||
|
if isinstance(path, str):
|
||||||
|
touched.add(path)
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=call.call_id,
|
||||||
|
result=outcome.result,
|
||||||
|
success=outcome.success,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Commit each turn's work onto the delegation branch, so that
|
||||||
|
# discarding the branch really does discard the work, and so a
|
||||||
|
# crash leaves the branch at a known commit (A3).
|
||||||
|
self._commit_turn(turn)
|
||||||
|
|
||||||
|
return DelegationResult(
|
||||||
|
"capped",
|
||||||
|
f"turn cap of {self._caps.turns} reached without a handback",
|
||||||
|
branch=branch,
|
||||||
|
artifacts=sorted(touched),
|
||||||
|
)
|
||||||
|
|
||||||
|
def _commit_turn(self, turn: int) -> None:
|
||||||
|
try:
|
||||||
|
self._git.commit_all(f"Delegation turn {turn + 1}")
|
||||||
|
except GitError:
|
||||||
|
# Losing a checkpoint commit does not invalidate the work; the
|
||||||
|
# canonical log still records what happened.
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _abandon(self, branch: str, base_branch: str) -> None:
|
||||||
|
try:
|
||||||
|
self._git.discard_branch(branch, back_to=base_branch)
|
||||||
|
except GitError:
|
||||||
|
# The branch is already recorded in the delegation events, so a
|
||||||
|
# failed cleanup leaves evidence rather than a silent mess.
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_handback(arguments: dict[str, Any]) -> Handback | None:
|
||||||
|
"""Validated locally rather than trusted: A14's rule for prompt-level
|
||||||
|
structured output applies to the handback too."""
|
||||||
|
required = ("changed", "now_true", "next")
|
||||||
|
if any(not str(arguments.get(key, "")).strip() for key in required):
|
||||||
|
return None
|
||||||
|
succeeded = arguments.get("succeeded")
|
||||||
|
if not isinstance(succeeded, bool):
|
||||||
|
return None
|
||||||
|
return Handback(
|
||||||
|
changed=str(arguments["changed"]).strip(),
|
||||||
|
now_true=str(arguments["now_true"]).strip(),
|
||||||
|
next=str(arguments["next"]).strip(),
|
||||||
|
succeeded=succeeded,
|
||||||
|
)
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
"""Local executor loop (spec A12 escalation triggers, A15 accommodations).
|
||||||
|
|
||||||
|
Runs the approved plan with the executor role — the cheap local model doing
|
||||||
|
mechanical work. Every tool call and result is checkpointed to the canonical
|
||||||
|
log as it happens (A13), so a crash mid-milestone leaves a resumable record.
|
||||||
|
|
||||||
|
Escalation is decided from outside the model (A12): consecutive command
|
||||||
|
failures and hard caps on turns, never the model's own claim that it is stuck
|
||||||
|
or finished-enough. Until delegation exists (step 7), an escalation gates to
|
||||||
|
the human with the failures attached rather than engaging a frontier model.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from nisse_dev.canonical import ToolCall, ToolResult
|
||||||
|
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import EXECUTOR_TOOLS, MUTATING_TOOLS, Workspace
|
||||||
|
|
||||||
|
EXECUTOR_SYSTEM = """You are nisse-dev's executor, doing mechanical work.
|
||||||
|
|
||||||
|
Follow the approved plan above exactly. It was written to be executed
|
||||||
|
verbatim; do not redesign it, expand its scope, or skip steps.
|
||||||
|
|
||||||
|
Work by calling tools. Verify your work by running the project's tests.
|
||||||
|
When every step of the plan is done and the tests pass, reply with a short
|
||||||
|
plain-text summary and no tool call — that ends the milestone."""
|
||||||
|
|
||||||
|
DEFAULT_MAX_TURNS = 24
|
||||||
|
DEFAULT_FAILURE_LIMIT = 2
|
||||||
|
|
||||||
|
# A turn with no tool call means "the plan is done". Believing that before
|
||||||
|
# any real work has happened would record an empty milestone as a success, so
|
||||||
|
# the claim is challenged once first.
|
||||||
|
_NUDGE = (
|
||||||
|
"\n\nYou have not made any change yet, so the plan cannot be complete. "
|
||||||
|
"Carry out the next step of the plan with a tool call, or run the tests "
|
||||||
|
"to show the work is already done."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ExecutorOutcome:
|
||||||
|
status: Literal["completed", "capped", "escalate"]
|
||||||
|
note: str
|
||||||
|
|
||||||
|
|
||||||
|
class LLMExecutor:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
client: LLMClient,
|
||||||
|
store: StateStore,
|
||||||
|
roles: RoleMap,
|
||||||
|
workspace: Workspace,
|
||||||
|
*,
|
||||||
|
max_turns: int = DEFAULT_MAX_TURNS,
|
||||||
|
failure_limit: int = DEFAULT_FAILURE_LIMIT,
|
||||||
|
) -> None:
|
||||||
|
self._client = client
|
||||||
|
self._store = store
|
||||||
|
self._roles = roles
|
||||||
|
self._workspace = workspace
|
||||||
|
self._max_turns = max_turns
|
||||||
|
self._failure_limit = failure_limit
|
||||||
|
|
||||||
|
def run(self, milestone: int) -> ExecutorOutcome:
|
||||||
|
model = self._roles.resolve("executor")
|
||||||
|
system = (
|
||||||
|
f"{EXECUTOR_SYSTEM}\n\n{self._workspace.describe()}"
|
||||||
|
f"\n\nYou are executing milestone {milestone}."
|
||||||
|
)
|
||||||
|
consecutive_failures = 0
|
||||||
|
did_work = False
|
||||||
|
challenged = False
|
||||||
|
|
||||||
|
for _turn in range(self._max_turns):
|
||||||
|
try:
|
||||||
|
produced = self._client.complete(
|
||||||
|
self._store.read_events(),
|
||||||
|
tools=EXECUTOR_TOOLS,
|
||||||
|
model=model,
|
||||||
|
system=system,
|
||||||
|
)
|
||||||
|
except LLMError as exc:
|
||||||
|
return ExecutorOutcome("escalate", f"executor call failed: {exc}")
|
||||||
|
|
||||||
|
calls: list[ToolCall] = []
|
||||||
|
for event in produced:
|
||||||
|
self._store.append_event(event)
|
||||||
|
if isinstance(event, ToolCall):
|
||||||
|
calls.append(event)
|
||||||
|
|
||||||
|
if not calls:
|
||||||
|
if did_work:
|
||||||
|
return ExecutorOutcome(
|
||||||
|
"completed", "executor reported the plan done"
|
||||||
|
)
|
||||||
|
if not challenged:
|
||||||
|
# Claiming completion without having touched anything is
|
||||||
|
# not evidence of completion; ask once before believing it.
|
||||||
|
challenged = True
|
||||||
|
system += _NUDGE
|
||||||
|
continue
|
||||||
|
return ExecutorOutcome(
|
||||||
|
"escalate",
|
||||||
|
"executor claimed the milestone was done without making any change",
|
||||||
|
)
|
||||||
|
|
||||||
|
for call in calls:
|
||||||
|
if call.tool in MUTATING_TOOLS:
|
||||||
|
did_work = True
|
||||||
|
outcome = self._workspace.dispatch(call.tool, call.arguments)
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=call.call_id,
|
||||||
|
result=outcome.result,
|
||||||
|
success=outcome.success,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
# Only commands count toward escalation: a failed read is the
|
||||||
|
# model looking around, a failed command is work not working.
|
||||||
|
if call.tool == "run_command":
|
||||||
|
consecutive_failures = (
|
||||||
|
0 if outcome.success else consecutive_failures + 1
|
||||||
|
)
|
||||||
|
if consecutive_failures >= self._failure_limit:
|
||||||
|
return ExecutorOutcome(
|
||||||
|
"escalate",
|
||||||
|
f"{consecutive_failures} consecutive command failures; "
|
||||||
|
f"last: {outcome.result[:500]}",
|
||||||
|
)
|
||||||
|
|
||||||
|
return ExecutorOutcome(
|
||||||
|
"capped", f"hit the {self._max_turns}-turn cap for milestone {milestone}"
|
||||||
|
)
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
"""Git operations for the agent's workspace (spec B2 outbound).
|
||||||
|
|
||||||
|
Push authentication uses the Gitea service-account token embedded in the
|
||||||
|
remote URL. The token is never written to disk in the repo config and is
|
||||||
|
scrubbed from anything this module raises or returns, so a failed push cannot
|
||||||
|
leak the credential into a transcript, a log line, or a PR comment.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
TIMEOUT_S = 120
|
||||||
|
|
||||||
|
|
||||||
|
class GitError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GitRepo:
|
||||||
|
root: Path
|
||||||
|
|
||||||
|
def _run(self, *args: str, secret: str | None = None) -> str:
|
||||||
|
try:
|
||||||
|
completed = subprocess.run(
|
||||||
|
["git", *args],
|
||||||
|
cwd=self.root,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=TIMEOUT_S,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||||
|
raise GitError(f"git {args[0]} failed: {_scrub(str(exc), secret)}") from exc
|
||||||
|
if completed.returncode != 0:
|
||||||
|
raise GitError(
|
||||||
|
f"git {args[0]} failed: "
|
||||||
|
f"{_scrub(completed.stderr.strip() or completed.stdout.strip(), secret)}"
|
||||||
|
)
|
||||||
|
return completed.stdout.strip()
|
||||||
|
|
||||||
|
# -- inspection -------------------------------------------------------
|
||||||
|
|
||||||
|
def current_branch(self) -> str:
|
||||||
|
# --show-current rather than rev-parse: it answers correctly on a
|
||||||
|
# freshly initialised repo, where HEAD points at an unborn branch.
|
||||||
|
return self._run("branch", "--show-current")
|
||||||
|
|
||||||
|
def head_sha(self) -> str:
|
||||||
|
return self._run("rev-parse", "HEAD")
|
||||||
|
|
||||||
|
def has_changes(self) -> bool:
|
||||||
|
return bool(self._run("status", "--porcelain"))
|
||||||
|
|
||||||
|
# -- mutation ---------------------------------------------------------
|
||||||
|
|
||||||
|
def init(self, *, default_branch: str = "main") -> None:
|
||||||
|
self.root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._run("init", "-q", "-b", default_branch)
|
||||||
|
|
||||||
|
def configure_identity(self, name: str, email: str) -> None:
|
||||||
|
self._run("config", "user.name", name)
|
||||||
|
self._run("config", "user.email", email)
|
||||||
|
|
||||||
|
def checkout_new_branch(self, name: str) -> None:
|
||||||
|
self._run("checkout", "-q", "-b", name)
|
||||||
|
|
||||||
|
def commit_all(self, message: str) -> str | None:
|
||||||
|
"""Returns the new commit sha, or None when there was nothing to
|
||||||
|
commit — an empty milestone is a fact to report, not an error."""
|
||||||
|
if not self.has_changes():
|
||||||
|
return None
|
||||||
|
self._run("add", "-A")
|
||||||
|
self._run("commit", "-q", "-m", message)
|
||||||
|
return self.head_sha()
|
||||||
|
|
||||||
|
def push(self, remote_url: str, branch: str, *, token: str | None = None) -> None:
|
||||||
|
# The URL is passed per-invocation rather than stored as a remote, so
|
||||||
|
# the token never lands in .git/config.
|
||||||
|
self._run("push", "-q", remote_url, f"HEAD:refs/heads/{branch}", secret=token)
|
||||||
|
|
||||||
|
def discard_branch(self, branch: str, *, back_to: str) -> None:
|
||||||
|
"""Throw away a branch and everything on it.
|
||||||
|
|
||||||
|
Used when a delegation aborts: A13 gives a delegation the same
|
||||||
|
atomicity that reconciliation gives an interrupted transition, and
|
||||||
|
half-finished work must not survive to be built upon.
|
||||||
|
"""
|
||||||
|
self._run("checkout", "-q", "--force", back_to)
|
||||||
|
self._run("branch", "-q", "-D", branch)
|
||||||
|
|
||||||
|
def delete_remote_branch(
|
||||||
|
self, remote_url: str, branch: str, *, token: str | None = None
|
||||||
|
) -> None:
|
||||||
|
self._run("push", "-q", remote_url, f":refs/heads/{branch}", secret=token)
|
||||||
|
|
||||||
|
|
||||||
|
def authenticated_url(base_url: str, repo: str, user: str, token: str) -> str:
|
||||||
|
host = base_url.split("://", 1)[-1].rstrip("/")
|
||||||
|
scheme = base_url.split("://", 1)[0] if "://" in base_url else "https"
|
||||||
|
return f"{scheme}://{user}:{token}@{host}/{repo}.git"
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub(text: str, secret: str | None) -> str:
|
||||||
|
return text.replace(secret, "***") if secret else text
|
||||||
@@ -0,0 +1,193 @@
|
|||||||
|
"""Gitea client (spec B2).
|
||||||
|
|
||||||
|
Outbound: open PRs and post comments through a dedicated service account with
|
||||||
|
a scoped token. Inbound is deliberately *polled* rather than webhook-driven,
|
||||||
|
so the agent VM stays outbound-only (B3), and intake is structured JSON —
|
||||||
|
never routed through a chat bridge.
|
||||||
|
|
||||||
|
``find_pull_request`` is the external-truth query that makes opening a PR
|
||||||
|
idempotent across a crash: boot reconciliation (A3) asks Gitea whether the
|
||||||
|
effect already happened rather than repeating it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from nisse_dev.reconcile import Reconciler
|
||||||
|
from nisse_dev.state import Intent
|
||||||
|
|
||||||
|
PR_PAGE_SIZE = 50
|
||||||
|
MAX_PR_PAGES = 40 # 2000 PRs; beyond that, say so rather than guess
|
||||||
|
|
||||||
|
|
||||||
|
class GiteaError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PullRequest:
|
||||||
|
number: int
|
||||||
|
state: str
|
||||||
|
head: str
|
||||||
|
url: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Comment:
|
||||||
|
id: int
|
||||||
|
author: str
|
||||||
|
body: str
|
||||||
|
created_at: str
|
||||||
|
|
||||||
|
|
||||||
|
class GiteaClient:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
base_url: str,
|
||||||
|
token: str,
|
||||||
|
*,
|
||||||
|
timeout_s: float = 30.0,
|
||||||
|
transport: httpx.BaseTransport | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._http = httpx.Client(
|
||||||
|
base_url=f"{base_url.rstrip('/')}/api/v1",
|
||||||
|
headers={"Authorization": f"token {token}"},
|
||||||
|
timeout=timeout_s,
|
||||||
|
transport=transport,
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_env(cls, env: Mapping[str, str] = os.environ) -> "GiteaClient":
|
||||||
|
base_url = env.get("NISSE_GITEA_URL")
|
||||||
|
token = env.get("NISSE_GITEA_TOKEN")
|
||||||
|
if not base_url or not token:
|
||||||
|
raise GiteaError("set NISSE_GITEA_URL and NISSE_GITEA_TOKEN")
|
||||||
|
return cls(base_url, token)
|
||||||
|
|
||||||
|
def _request(self, method: str, path: str, **kwargs: Any) -> Any:
|
||||||
|
try:
|
||||||
|
response = self._http.request(method, path, **kwargs)
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise GiteaError(f"gitea request failed: {exc}") from exc
|
||||||
|
if response.status_code >= 400:
|
||||||
|
raise GiteaError(
|
||||||
|
f"gitea {method} {path} returned {response.status_code}: "
|
||||||
|
f"{response.text[:300]}"
|
||||||
|
)
|
||||||
|
return response.json() if response.content else None
|
||||||
|
|
||||||
|
def whoami(self) -> str:
|
||||||
|
return str(self._request("GET", "/user")["login"])
|
||||||
|
|
||||||
|
def open_pull_request(
|
||||||
|
self, repo: str, *, head: str, base: str, title: str, body: str
|
||||||
|
) -> PullRequest:
|
||||||
|
data = self._request(
|
||||||
|
"POST",
|
||||||
|
f"/repos/{repo}/pulls",
|
||||||
|
json={"head": head, "base": base, "title": title, "body": body},
|
||||||
|
)
|
||||||
|
return _pull_request(data)
|
||||||
|
|
||||||
|
def find_pull_request(self, repo: str, head: str) -> PullRequest | None:
|
||||||
|
"""External truth for A3: did the PR for this branch already get
|
||||||
|
opened? Includes closed PRs — a crash after opening must not be
|
||||||
|
answered with "no PR is open, so open another".
|
||||||
|
|
||||||
|
Pages to the end rather than sampling the first page: a "not found"
|
||||||
|
that only means "not in the newest 50" would licence a duplicate PR.
|
||||||
|
If the history is longer than this walk, raise instead of guessing —
|
||||||
|
the reconciler turns that into "unverifiable" and gates to the human.
|
||||||
|
"""
|
||||||
|
for page in range(1, MAX_PR_PAGES + 1):
|
||||||
|
data = self._request(
|
||||||
|
"GET",
|
||||||
|
f"/repos/{repo}/pulls",
|
||||||
|
params={"state": "all", "limit": PR_PAGE_SIZE, "page": page},
|
||||||
|
)
|
||||||
|
items = data or []
|
||||||
|
for item in items:
|
||||||
|
if (item.get("head") or {}).get("ref") == head:
|
||||||
|
return _pull_request(item)
|
||||||
|
if len(items) < PR_PAGE_SIZE:
|
||||||
|
return None # reached the end: definitively absent
|
||||||
|
raise GiteaError(
|
||||||
|
f"searched {MAX_PR_PAGES * PR_PAGE_SIZE} pull requests in {repo} "
|
||||||
|
f"without reaching the end; cannot confirm whether {head!r} was "
|
||||||
|
"already opened"
|
||||||
|
)
|
||||||
|
|
||||||
|
def list_comments(
|
||||||
|
self, repo: str, index: int, since: str | None = None
|
||||||
|
) -> list[Comment]:
|
||||||
|
params: dict[str, str] = {}
|
||||||
|
if since is not None:
|
||||||
|
params["since"] = since
|
||||||
|
data = self._request(
|
||||||
|
"GET", f"/repos/{repo}/issues/{index}/comments", params=params
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
Comment(
|
||||||
|
id=item["id"],
|
||||||
|
author=(item.get("user") or {}).get("login", ""),
|
||||||
|
body=item.get("body", ""),
|
||||||
|
created_at=item.get("created_at", ""),
|
||||||
|
)
|
||||||
|
for item in data or []
|
||||||
|
]
|
||||||
|
|
||||||
|
def post_comment(self, repo: str, index: int, body: str) -> Comment:
|
||||||
|
data = self._request(
|
||||||
|
"POST", f"/repos/{repo}/issues/{index}/comments", json={"body": body}
|
||||||
|
)
|
||||||
|
return Comment(
|
||||||
|
id=data["id"],
|
||||||
|
author=(data.get("user") or {}).get("login", ""),
|
||||||
|
body=data.get("body", ""),
|
||||||
|
created_at=data.get("created_at", ""),
|
||||||
|
)
|
||||||
|
|
||||||
|
def create_issue(self, repo: str, title: str, body: str) -> int:
|
||||||
|
data = self._request(
|
||||||
|
"POST", f"/repos/{repo}/issues", json={"title": title, "body": body}
|
||||||
|
)
|
||||||
|
return int(data["number"])
|
||||||
|
|
||||||
|
def close_pull_request(self, repo: str, index: int) -> None:
|
||||||
|
self._request("PATCH", f"/repos/{repo}/pulls/{index}", json={"state": "closed"})
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._http.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _pull_request(data: dict[str, Any]) -> PullRequest:
|
||||||
|
return PullRequest(
|
||||||
|
number=data["number"],
|
||||||
|
state=data.get("state", ""),
|
||||||
|
head=(data.get("head") or {}).get("ref", ""),
|
||||||
|
url=data.get("html_url", ""),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def pull_request_reconciler(client: GiteaClient, repo: str) -> Reconciler:
|
||||||
|
"""Reconciler for an ``open_pr`` intent (A3 step 4).
|
||||||
|
|
||||||
|
An unreachable Gitea yields None — unverifiable, so the agent gates to
|
||||||
|
the human rather than risking a duplicate PR. That is the conservative
|
||||||
|
direction: a delayed decision costs nothing, a repeated side effect does.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def check(intent: Intent) -> bool | None:
|
||||||
|
branch = intent.payload.get("branch")
|
||||||
|
if not branch:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return client.find_pull_request(repo, branch) is not None
|
||||||
|
except GiteaError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return check
|
||||||
@@ -12,6 +12,7 @@ config change cannot split a task across two planners.
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import time
|
||||||
from collections.abc import Mapping
|
from collections.abc import Mapping
|
||||||
from dataclasses import asdict, dataclass
|
from dataclasses import asdict, dataclass
|
||||||
from typing import Any, ClassVar, Protocol
|
from typing import Any, ClassVar, Protocol
|
||||||
@@ -26,12 +27,21 @@ from nisse_dev.canonical import (
|
|||||||
mint_call_id,
|
mint_call_id,
|
||||||
)
|
)
|
||||||
from nisse_dev.render import OpenAIChatRenderer
|
from nisse_dev.render import OpenAIChatRenderer
|
||||||
|
from nisse_dev.render_anthropic import AnthropicMessagesRenderer
|
||||||
|
|
||||||
|
|
||||||
class LLMError(RuntimeError):
|
class LLMError(RuntimeError):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# A rate limit is a transient condition, not a bad request. Losing a
|
||||||
|
# ten-turn delegation to one wastes far more tokens than waiting out the
|
||||||
|
# window, so retry once before surfacing it (A13's caps still bound the
|
||||||
|
# total).
|
||||||
|
RATE_LIMIT_STATUS = 429
|
||||||
|
RATE_LIMIT_RETRY_S = 20.0
|
||||||
|
|
||||||
|
|
||||||
class LLMClient(Protocol):
|
class LLMClient(Protocol):
|
||||||
def complete(
|
def complete(
|
||||||
self,
|
self,
|
||||||
@@ -81,6 +91,7 @@ class LiteLLMClient:
|
|||||||
*,
|
*,
|
||||||
timeout_s: float = 120.0,
|
timeout_s: float = 120.0,
|
||||||
transport: httpx.BaseTransport | None = None,
|
transport: httpx.BaseTransport | None = None,
|
||||||
|
sleep: Any = time.sleep,
|
||||||
) -> None:
|
) -> None:
|
||||||
self._http = httpx.Client(
|
self._http = httpx.Client(
|
||||||
base_url=base_url.rstrip("/"),
|
base_url=base_url.rstrip("/"),
|
||||||
@@ -89,6 +100,7 @@ class LiteLLMClient:
|
|||||||
transport=transport,
|
transport=transport,
|
||||||
)
|
)
|
||||||
self._renderer = OpenAIChatRenderer()
|
self._renderer = OpenAIChatRenderer()
|
||||||
|
self._sleep = sleep
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_env(cls, env: Mapping[str, str] = os.environ) -> "LiteLLMClient":
|
def from_env(cls, env: Mapping[str, str] = os.environ) -> "LiteLLMClient":
|
||||||
@@ -106,10 +118,9 @@ class LiteLLMClient:
|
|||||||
system: str | None = None,
|
system: str | None = None,
|
||||||
) -> list[CanonicalEvent]:
|
) -> list[CanonicalEvent]:
|
||||||
body = self._renderer.render(events, tools, model, system=system)
|
body = self._renderer.render(events, tools, model, system=system)
|
||||||
try:
|
response = _post_with_rate_limit_retry(
|
||||||
response = self._http.post("/chat/completions", json=body)
|
self._http, "/chat/completions", body, self._sleep
|
||||||
except httpx.HTTPError as exc:
|
)
|
||||||
raise LLMError(f"LLM request failed: {exc}") from exc
|
|
||||||
if response.status_code != 200:
|
if response.status_code != 200:
|
||||||
raise LLMError(
|
raise LLMError(
|
||||||
f"LLM request returned {response.status_code}: {response.text[:500]}"
|
f"LLM request returned {response.status_code}: {response.text[:500]}"
|
||||||
@@ -157,3 +168,111 @@ class LiteLLMClient:
|
|||||||
|
|
||||||
def close(self) -> None:
|
def close(self) -> None:
|
||||||
self._http.close()
|
self._http.close()
|
||||||
|
|
||||||
|
|
||||||
|
class AnthropicPassthroughClient:
|
||||||
|
"""LiteLLM's Anthropic passthrough (A5). Same interface as LiteLLMClient,
|
||||||
|
so a caller chooses a route rather than a different kind of client.
|
||||||
|
|
||||||
|
Delegations take this path because fidelity matters there (A13);
|
||||||
|
ask_expert and fallbacks use the normalized route, where it does not.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
base_url: str,
|
||||||
|
token: str,
|
||||||
|
*,
|
||||||
|
timeout_s: float = 300.0,
|
||||||
|
max_tokens: int = 4096,
|
||||||
|
transport: httpx.BaseTransport | None = None,
|
||||||
|
sleep: Any = time.sleep,
|
||||||
|
) -> None:
|
||||||
|
self._http = httpx.Client(
|
||||||
|
base_url=base_url.rstrip("/"),
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"x-api-key": token,
|
||||||
|
"anthropic-version": "2023-06-01",
|
||||||
|
},
|
||||||
|
timeout=timeout_s,
|
||||||
|
transport=transport,
|
||||||
|
)
|
||||||
|
self._renderer = AnthropicMessagesRenderer()
|
||||||
|
self._max_tokens = max_tokens
|
||||||
|
self._sleep = sleep
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_env(
|
||||||
|
cls, env: Mapping[str, str] = os.environ
|
||||||
|
) -> "AnthropicPassthroughClient":
|
||||||
|
base_url = env.get("NISSE_LLM_BASE_URL")
|
||||||
|
token = env.get("NISSE_LLM_TOKEN")
|
||||||
|
if not base_url or not token:
|
||||||
|
raise LLMError("set NISSE_LLM_BASE_URL and NISSE_LLM_TOKEN")
|
||||||
|
return cls(base_url, token)
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
events: list[StoredEvent],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
system: str | None = None,
|
||||||
|
) -> list[CanonicalEvent]:
|
||||||
|
body = self._renderer.render(
|
||||||
|
events, tools, model, system=system, max_tokens=self._max_tokens
|
||||||
|
)
|
||||||
|
response = _post_with_rate_limit_retry(
|
||||||
|
self._http, "/v1/messages", body, self._sleep
|
||||||
|
)
|
||||||
|
if response.status_code != 200:
|
||||||
|
raise LLMError(
|
||||||
|
f"passthrough returned {response.status_code}: {response.text[:500]}"
|
||||||
|
)
|
||||||
|
return self._parse(response.json(), requested_model=model)
|
||||||
|
|
||||||
|
def _parse(
|
||||||
|
self, data: dict[str, Any], requested_model: str
|
||||||
|
) -> list[CanonicalEvent]:
|
||||||
|
content = data.get("content")
|
||||||
|
if not isinstance(content, list):
|
||||||
|
raise LLMError(f"malformed passthrough response: {str(data)[:300]}")
|
||||||
|
produced_by = data.get("model") or requested_model
|
||||||
|
out: list[CanonicalEvent] = []
|
||||||
|
for block in content:
|
||||||
|
kind = block.get("type")
|
||||||
|
if kind == "text" and block.get("text"):
|
||||||
|
out.append(AssistantText(text=block["text"], model=produced_by))
|
||||||
|
elif kind == "tool_use":
|
||||||
|
# Provider ids are mapped at the boundary and never used as a
|
||||||
|
# join key (A10); the canonical log owns identity.
|
||||||
|
out.append(
|
||||||
|
ToolCall(
|
||||||
|
call_id=mint_call_id(),
|
||||||
|
tool=block["name"],
|
||||||
|
arguments=block.get("input") or {},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
# Vendor-opaque reasoning blocks (thinking, redacted_thinking) are
|
||||||
|
# deliberately dropped for now: A10 requires storing them tagged
|
||||||
|
# with the producing model and never replaying them outside the
|
||||||
|
# session that made them, and the canonical schema has no event
|
||||||
|
# for that yet. Nothing requests extended thinking, so none arrive.
|
||||||
|
return out
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._http.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _post_with_rate_limit_retry(
|
||||||
|
http: httpx.Client, path: str, body: dict[str, Any], sleep: Any
|
||||||
|
) -> httpx.Response:
|
||||||
|
for attempt in (1, 2):
|
||||||
|
try:
|
||||||
|
response = http.post(path, json=body)
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
raise LLMError(f"LLM request failed: {exc}") from exc
|
||||||
|
if response.status_code != RATE_LIMIT_STATUS or attempt == 2:
|
||||||
|
return response
|
||||||
|
sleep(RATE_LIMIT_RETRY_S)
|
||||||
|
raise AssertionError("unreachable")
|
||||||
@@ -11,7 +11,7 @@ crash mid-planning leaves a truthful transcript.
|
|||||||
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from nisse_dev.canonical import AssistantText, ToolCall
|
from nisse_dev.canonical import AssistantText, ToolCall, ToolResult
|
||||||
from nisse_dev.control import AskHuman, PlanningError, SubmitPlan
|
from nisse_dev.control import AskHuman, PlanningError, SubmitPlan
|
||||||
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
from nisse_dev.llm import LLMClient, LLMError, RoleMap
|
||||||
from nisse_dev.state import StateStore
|
from nisse_dev.state import StateStore
|
||||||
@@ -142,6 +142,21 @@ class LLMPlanningAgent:
|
|||||||
continue
|
continue
|
||||||
if isinstance(event, ToolCall):
|
if isinstance(event, ToolCall):
|
||||||
action = self._interpret(event)
|
action = self._interpret(event)
|
||||||
|
if action is not None:
|
||||||
|
# The gate is this call's outcome. Recording it keeps the
|
||||||
|
# transcript renderable: providers reject a tool call with
|
||||||
|
# no result, and the next planning turn replays this log.
|
||||||
|
self._store.append_event(
|
||||||
|
ToolResult(
|
||||||
|
call_id=event.call_id,
|
||||||
|
result=(
|
||||||
|
"plan posted for approval; turn ended"
|
||||||
|
if isinstance(action, SubmitPlan)
|
||||||
|
else "question posted; awaiting the operator"
|
||||||
|
),
|
||||||
|
success=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
return action
|
return action
|
||||||
|
|
||||||
def _interpret(self, call: ToolCall) -> SubmitPlan | AskHuman | None:
|
def _interpret(self, call: ToolCall) -> SubmitPlan | AskHuman | None:
|
||||||
|
|||||||
@@ -0,0 +1,104 @@
|
|||||||
|
"""Gitea comment intake (spec B2).
|
||||||
|
|
||||||
|
Polling, not webhooks: the agent VM stays outbound-only (B3), so nothing
|
||||||
|
needs to reach in. Intake is structured JSON straight from the API and never
|
||||||
|
routed through a chat bridge.
|
||||||
|
|
||||||
|
Outage posture, per B2: back off exponentially, cap at a few minutes, retry
|
||||||
|
indefinitely, and log. Polling is pure intake — an outage delays a reaction
|
||||||
|
but must never fail a task or force a state transition.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import threading
|
||||||
|
from collections.abc import Callable, Iterable
|
||||||
|
|
||||||
|
from nisse_dev.gitea import Comment, GiteaClient, GiteaError
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
|
||||||
|
DEFAULT_INTERVAL_S = 20.0
|
||||||
|
INITIAL_BACKOFF_S = 5.0
|
||||||
|
MAX_BACKOFF_S = 300.0
|
||||||
|
|
||||||
|
|
||||||
|
class CommentPoller:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
client: GiteaClient,
|
||||||
|
repo: str,
|
||||||
|
on_comment: Callable[[Comment], None],
|
||||||
|
*,
|
||||||
|
oplog: OpLog | None = None,
|
||||||
|
ignore_authors: Iterable[str] = (),
|
||||||
|
interval_s: float = DEFAULT_INTERVAL_S,
|
||||||
|
initial_backoff_s: float = INITIAL_BACKOFF_S,
|
||||||
|
max_backoff_s: float = MAX_BACKOFF_S,
|
||||||
|
) -> None:
|
||||||
|
self._client = client
|
||||||
|
self._repo = repo
|
||||||
|
self._on_comment = on_comment
|
||||||
|
self._oplog = oplog
|
||||||
|
self._ignore = set(ignore_authors)
|
||||||
|
self._interval_s = interval_s
|
||||||
|
self._initial_backoff_s = initial_backoff_s
|
||||||
|
self._max_backoff_s = max_backoff_s
|
||||||
|
self._seen: set[int] = set()
|
||||||
|
self._stop = threading.Event()
|
||||||
|
self._thread: threading.Thread | None = None
|
||||||
|
|
||||||
|
def prime(self, index: int) -> None:
|
||||||
|
"""Mark existing comments as seen without delivering them.
|
||||||
|
|
||||||
|
A poller that starts by replaying the whole thread would re-act to
|
||||||
|
feedback already addressed — including its own earlier replies.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
for comment in self._client.list_comments(self._repo, index):
|
||||||
|
self._seen.add(comment.id)
|
||||||
|
except GiteaError:
|
||||||
|
# Nothing seen means the first successful poll delivers the
|
||||||
|
# backlog once. Better than refusing to start.
|
||||||
|
pass
|
||||||
|
|
||||||
|
def poll_once(self, index: int) -> list[Comment]:
|
||||||
|
"""One pass; returns what was newly delivered. Separated from the
|
||||||
|
loop so behaviour is testable without threads or sleeps."""
|
||||||
|
delivered: list[Comment] = []
|
||||||
|
for comment in self._client.list_comments(self._repo, index):
|
||||||
|
if comment.id in self._seen:
|
||||||
|
continue
|
||||||
|
self._seen.add(comment.id)
|
||||||
|
if comment.author in self._ignore:
|
||||||
|
continue # the agent's own replies are not new input
|
||||||
|
delivered.append(comment)
|
||||||
|
self._on_comment(comment)
|
||||||
|
return delivered
|
||||||
|
|
||||||
|
def start(self, index: int) -> None:
|
||||||
|
self._stop.clear()
|
||||||
|
self._thread = threading.Thread(target=self._loop, args=(index,), daemon=True)
|
||||||
|
self._thread.start()
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self._stop.set()
|
||||||
|
if self._thread is not None:
|
||||||
|
self._thread.join(timeout=self._interval_s + 5)
|
||||||
|
self._thread = None
|
||||||
|
|
||||||
|
def _loop(self, index: int) -> None:
|
||||||
|
backoff = self._initial_backoff_s
|
||||||
|
while not self._stop.is_set():
|
||||||
|
try:
|
||||||
|
self.poll_once(index)
|
||||||
|
except GiteaError as exc:
|
||||||
|
if self._oplog is not None:
|
||||||
|
self._oplog.emit(
|
||||||
|
"gitea.poll_failed",
|
||||||
|
cause="gitea unreachable",
|
||||||
|
error=str(exc)[:300],
|
||||||
|
retry_in_s=backoff,
|
||||||
|
)
|
||||||
|
self._stop.wait(backoff)
|
||||||
|
backoff = min(backoff * 2, self._max_backoff_s)
|
||||||
|
continue
|
||||||
|
backoff = self._initial_backoff_s
|
||||||
|
self._stop.wait(self._interval_s)
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
"""Publishing a finished milestone as a Gitea PR (spec B2, A2/A3).
|
||||||
|
|
||||||
|
The ordering here is the whole point. Intent is recorded *before* the first
|
||||||
|
externally visible act, so a crash anywhere in the sequence leaves a durable
|
||||||
|
record of what was in flight. Boot reconciliation then asks Gitea whether the
|
||||||
|
PR exists rather than repeating the push-and-open — the spec's own example of
|
||||||
|
an idempotent transition reconciled against external truth.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from nisse_dev.git import GitRepo, authenticated_url
|
||||||
|
from nisse_dev.gitea import (
|
||||||
|
GiteaClient,
|
||||||
|
GiteaError,
|
||||||
|
PullRequest,
|
||||||
|
pull_request_reconciler,
|
||||||
|
)
|
||||||
|
from nisse_dev.reconcile import Reconciler
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
|
||||||
|
OPEN_PR_INTENT = "open_pr"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PublishResult:
|
||||||
|
pull_request: PullRequest | None
|
||||||
|
note: str
|
||||||
|
|
||||||
|
|
||||||
|
class MilestonePublisher:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
store: StateStore,
|
||||||
|
client: GiteaClient,
|
||||||
|
repo: str,
|
||||||
|
git_repo: GitRepo,
|
||||||
|
*,
|
||||||
|
base_url: str,
|
||||||
|
account: str,
|
||||||
|
token: str,
|
||||||
|
base_branch: str = "main",
|
||||||
|
) -> None:
|
||||||
|
self._store = store
|
||||||
|
self._client = client
|
||||||
|
self._repo = repo
|
||||||
|
self._git = git_repo
|
||||||
|
self._base_branch = base_branch
|
||||||
|
self._remote = authenticated_url(base_url, repo, account, token)
|
||||||
|
self._token = token
|
||||||
|
|
||||||
|
def reconcilers(self) -> dict[str, Reconciler]:
|
||||||
|
return {OPEN_PR_INTENT: pull_request_reconciler(self._client, self._repo)}
|
||||||
|
|
||||||
|
def branch_for(self, task_id: str, milestone: int) -> str:
|
||||||
|
# Namespaced by task so re-running a project against the same repo
|
||||||
|
# cannot collide with an earlier run's branch.
|
||||||
|
return f"{task_id}/milestone-{milestone}"
|
||||||
|
|
||||||
|
def publish(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
task_id: str,
|
||||||
|
milestone: int,
|
||||||
|
project: str,
|
||||||
|
plan: str,
|
||||||
|
service_url: str | None = None,
|
||||||
|
) -> PublishResult:
|
||||||
|
branch = self.branch_for(task_id, milestone)
|
||||||
|
if self._git.current_branch() != branch:
|
||||||
|
self._git.checkout_new_branch(branch)
|
||||||
|
sha = self._git.commit_all(f"Milestone {milestone}: {project}")
|
||||||
|
if sha is None:
|
||||||
|
# Not an error: the executor may have concluded there was nothing
|
||||||
|
# to change. Report it and let the operator decide.
|
||||||
|
return PublishResult(None, "no changes to publish")
|
||||||
|
|
||||||
|
intent_id = self._store.begin_intent(
|
||||||
|
OPEN_PR_INTENT,
|
||||||
|
{"branch": branch, "milestone": milestone, "sha": sha},
|
||||||
|
cause=f"publishing milestone {milestone}",
|
||||||
|
)
|
||||||
|
self._git.push(self._remote, branch, token=self._token)
|
||||||
|
pr = self._client.open_pull_request(
|
||||||
|
self._repo,
|
||||||
|
head=branch,
|
||||||
|
base=self._base_branch,
|
||||||
|
title=f"Milestone {milestone}: {project}",
|
||||||
|
body=_pr_body(milestone, plan, service_url),
|
||||||
|
)
|
||||||
|
self._store.commit_intent(intent_id, cause=f"opened PR #{pr.number}")
|
||||||
|
return PublishResult(pr, f"opened PR #{pr.number}: {pr.url}")
|
||||||
|
|
||||||
|
def comment(self, index: int, body: str) -> None:
|
||||||
|
"""Reply on the PR. Courtesy only — the pushed commit is the real
|
||||||
|
answer — so a failure here must not derail the review loop."""
|
||||||
|
try:
|
||||||
|
self._client.post_comment(self._repo, index, body)
|
||||||
|
except GiteaError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def push_followup(
|
||||||
|
self, *, task_id: str, milestone: int, message: str
|
||||||
|
) -> str | None:
|
||||||
|
"""Push a fix commit onto an open milestone's branch.
|
||||||
|
|
||||||
|
No intent is recorded: the PR already exists, so a crash mid-push
|
||||||
|
costs a repeated push of the same commit, which is harmless. That is
|
||||||
|
the difference from publish(), where the side effect is a new PR.
|
||||||
|
"""
|
||||||
|
branch = self.branch_for(task_id, milestone)
|
||||||
|
if self._git.current_branch() != branch:
|
||||||
|
self._git.checkout_new_branch(branch)
|
||||||
|
sha = self._git.commit_all(message)
|
||||||
|
if sha is None:
|
||||||
|
return None
|
||||||
|
self._git.push(self._remote, branch, token=self._token)
|
||||||
|
return sha
|
||||||
|
|
||||||
|
|
||||||
|
def _pr_body(milestone: int, plan: str, service_url: str | None) -> str:
|
||||||
|
parts = [
|
||||||
|
f"Milestone {milestone}, implemented by nisse-dev against the approved plan.",
|
||||||
|
"",
|
||||||
|
"## Approved plan",
|
||||||
|
"",
|
||||||
|
plan,
|
||||||
|
]
|
||||||
|
if service_url:
|
||||||
|
# B2: a runnable test service's address belongs in the PR description
|
||||||
|
# so a reviewer can exercise it. Nothing reports one yet; the browser
|
||||||
|
# tool (A6) is what will.
|
||||||
|
parts += ["", "## Running test service", "", service_url]
|
||||||
|
return "\n".join(parts)
|
||||||
@@ -57,8 +57,24 @@ class OpenAIChatRenderer:
|
|||||||
messages: list[dict[str, Any]] = []
|
messages: list[dict[str, Any]] = []
|
||||||
if system is not None:
|
if system is not None:
|
||||||
messages.append({"role": "system", "content": system})
|
messages.append({"role": "system", "content": system})
|
||||||
for stored in effective_events(events):
|
kept = effective_events(events)
|
||||||
|
answered = {e.event.call_id for e in kept if isinstance(e.event, ToolResult)}
|
||||||
|
for stored in kept:
|
||||||
messages.append(self._message_for(stored.event))
|
messages.append(self._message_for(stored.event))
|
||||||
|
# Providers reject a tool call with no result following it. One
|
||||||
|
# can legitimately be missing: a crash between the call and its
|
||||||
|
# result (A3), or a turn-ending gate tool. Render a placeholder so
|
||||||
|
# the transcript stays replayable rather than poisoned.
|
||||||
|
if isinstance(stored.event, ToolCall) and (
|
||||||
|
stored.event.call_id not in answered
|
||||||
|
):
|
||||||
|
messages.append(
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"tool_call_id": stored.event.call_id,
|
||||||
|
"content": "[no result recorded]",
|
||||||
|
}
|
||||||
|
)
|
||||||
body: dict[str, Any] = {"model": model, "messages": messages}
|
body: dict[str, Any] = {"model": model, "messages": messages}
|
||||||
if tools:
|
if tools:
|
||||||
body["tools"] = tools
|
body["tools"] = tools
|
||||||
|
|||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""Anthropic Messages renderer (spec A10, A13).
|
||||||
|
|
||||||
|
The second adapter, and the reason A13 puts normalisation in the renderer
|
||||||
|
rather than on the wire: the two vendors disagree *structurally*, not
|
||||||
|
cosmetically.
|
||||||
|
|
||||||
|
| | OpenAI-compatible | Anthropic Messages |
|
||||||
|
|---|---|---|
|
||||||
|
| system prompt | a message with role `system` | a top-level `system` field |
|
||||||
|
| tool call | `tool_calls` array on the message | a `tool_use` content block |
|
||||||
|
| tool result | a message with role `tool` | a `tool_result` block in a **user** message |
|
||||||
|
| tool schema | `function.parameters` | `input_schema` |
|
||||||
|
|
||||||
|
A middlebox that normalised between these would be lossy exactly where a
|
||||||
|
long-running delegation lives, which is why delegations take the passthrough
|
||||||
|
route and this renderer builds the vendor-native body.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from nisse_dev.canonical import (
|
||||||
|
AssistantText,
|
||||||
|
Compaction,
|
||||||
|
DelegationCompleted,
|
||||||
|
DelegationStarted,
|
||||||
|
GateOpened,
|
||||||
|
PlanSubmitted,
|
||||||
|
StoredEvent,
|
||||||
|
ToolCall,
|
||||||
|
ToolResult,
|
||||||
|
UserMessage,
|
||||||
|
)
|
||||||
|
from nisse_dev.render import effective_events
|
||||||
|
|
||||||
|
DEFAULT_MAX_TOKENS = 4096
|
||||||
|
NO_RESULT_PLACEHOLDER = "[no result recorded]"
|
||||||
|
|
||||||
|
|
||||||
|
def to_anthropic_tools(tools: list[dict[str, Any]] | None) -> list[dict[str, Any]]:
|
||||||
|
"""Convert the OpenAI-shaped manifests the tool modules already define,
|
||||||
|
so a tool is declared once and rendered per vendor."""
|
||||||
|
converted: list[dict[str, Any]] = []
|
||||||
|
for tool in tools or []:
|
||||||
|
fn = tool.get("function", tool)
|
||||||
|
converted.append(
|
||||||
|
{
|
||||||
|
"name": fn["name"],
|
||||||
|
"description": fn.get("description", ""),
|
||||||
|
"input_schema": fn.get(
|
||||||
|
"parameters", {"type": "object", "properties": {}}
|
||||||
|
),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return converted
|
||||||
|
|
||||||
|
|
||||||
|
class AnthropicMessagesRenderer:
|
||||||
|
def render(
|
||||||
|
self,
|
||||||
|
events: list[StoredEvent],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
system: str | None = None,
|
||||||
|
*,
|
||||||
|
max_tokens: int = DEFAULT_MAX_TOKENS,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
kept = effective_events(events)
|
||||||
|
answered = {e.event.call_id for e in kept if isinstance(e.event, ToolResult)}
|
||||||
|
blocks: list[tuple[str, dict[str, Any]]] = []
|
||||||
|
for stored in kept:
|
||||||
|
blocks.extend(self._blocks_for(stored.event))
|
||||||
|
# A dangling tool_use is rejected outright by this API — more
|
||||||
|
# strictly than the OpenAI shape — so a crash between a call and
|
||||||
|
# its result must not poison the transcript (A3).
|
||||||
|
if isinstance(stored.event, ToolCall) and (
|
||||||
|
stored.event.call_id not in answered
|
||||||
|
):
|
||||||
|
blocks.append(
|
||||||
|
(
|
||||||
|
"user",
|
||||||
|
{
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": stored.event.call_id,
|
||||||
|
"content": NO_RESULT_PLACEHOLDER,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
body: dict[str, Any] = {
|
||||||
|
"model": model,
|
||||||
|
"max_tokens": max_tokens,
|
||||||
|
"messages": _coalesce(blocks),
|
||||||
|
}
|
||||||
|
if system is not None:
|
||||||
|
body["system"] = system
|
||||||
|
if tools:
|
||||||
|
body["tools"] = to_anthropic_tools(tools)
|
||||||
|
return body
|
||||||
|
|
||||||
|
def _blocks_for(self, event: object) -> list[tuple[str, dict[str, Any]]]:
|
||||||
|
match event:
|
||||||
|
case UserMessage(text=text):
|
||||||
|
return [("user", {"type": "text", "text": text})]
|
||||||
|
case AssistantText(text=text):
|
||||||
|
return [("assistant", {"type": "text", "text": text})]
|
||||||
|
case ToolCall(call_id=call_id, tool=tool, arguments=arguments):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"assistant",
|
||||||
|
{
|
||||||
|
"type": "tool_use",
|
||||||
|
"id": call_id,
|
||||||
|
"name": tool,
|
||||||
|
"input": arguments,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case ToolResult(call_id=call_id, result=result):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"user",
|
||||||
|
{
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": call_id,
|
||||||
|
"content": result,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case PlanSubmitted(plan=plan):
|
||||||
|
return [("assistant", {"type": "text", "text": plan})]
|
||||||
|
case GateOpened(gate=gate, reason=reason):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"user",
|
||||||
|
{"type": "text", "text": f"[gate {gate} opened: {reason}]"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case DelegationStarted(brief=brief, model=model):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"assistant",
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": f"[delegation started ({model}): {brief}]",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case DelegationCompleted(handback=handback):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"assistant",
|
||||||
|
{"type": "text", "text": f"[delegation handback]\n{handback}"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case Compaction(summary=summary):
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"user",
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": f"[summary of earlier conversation]\n{summary}",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
case _:
|
||||||
|
raise ValueError(f"unrenderable event: {event!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _coalesce(
|
||||||
|
blocks: list[tuple[str, dict[str, Any]]],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Merge adjacent blocks of the same role into one message.
|
||||||
|
|
||||||
|
The Messages API expects alternating roles, and a tool_result must sit in
|
||||||
|
the user message directly after the assistant's tool_use — which happens
|
||||||
|
naturally once same-role runs are merged.
|
||||||
|
"""
|
||||||
|
messages: list[dict[str, Any]] = []
|
||||||
|
for role, block in blocks:
|
||||||
|
if messages and messages[-1]["role"] == role:
|
||||||
|
messages[-1]["content"].append(block)
|
||||||
|
else:
|
||||||
|
messages.append({"role": role, "content": [block]})
|
||||||
|
return messages
|
||||||
@@ -0,0 +1,235 @@
|
|||||||
|
"""Executor tools (spec A15: reduced schema, flat arguments).
|
||||||
|
|
||||||
|
Deliberately unsandboxed. The VM is the trust boundary (B3) and tool use is
|
||||||
|
auto-approved inside an approved unit of work (A4), so per-command
|
||||||
|
confinement would add failure modes without adding safety. The workspace is
|
||||||
|
where work happens — the default for relative paths and the command cwd — not
|
||||||
|
a wall. Protecting agent state from the agent is a mount-layout concern
|
||||||
|
(A2/B3), not this module's job.
|
||||||
|
|
||||||
|
What *is* enforced here is operational: commands cannot hang the loop, cannot
|
||||||
|
consume the context window with megabytes of build log, and do not inherit
|
||||||
|
the agent's own credentials.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
# Tools that change or verify something. Reading a file or listing a
|
||||||
|
# directory is a model looking around, not doing the work, so neither counts
|
||||||
|
# as evidence that work was carried out.
|
||||||
|
MUTATING_TOOLS = frozenset({"write_file", "run_command"})
|
||||||
|
|
||||||
|
DEFAULT_TIMEOUT_S = 120
|
||||||
|
MAX_OUTPUT_CHARS = 4000
|
||||||
|
|
||||||
|
EXECUTOR_TOOLS: list[dict[str, Any]] = [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "read_file",
|
||||||
|
"description": "Read a UTF-8 text file.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"path": {"type": "string"}},
|
||||||
|
"required": ["path"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "write_file",
|
||||||
|
"description": "Write a UTF-8 text file, creating parent directories.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"path": {"type": "string"},
|
||||||
|
"content": {"type": "string"},
|
||||||
|
},
|
||||||
|
"required": ["path", "content"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "list_dir",
|
||||||
|
"description": "List the entries of a directory.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"path": {"type": "string"}},
|
||||||
|
"required": ["path"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "run_command",
|
||||||
|
"description": (
|
||||||
|
"Run a shell command in the workspace. Returns its exit code "
|
||||||
|
"and output; a non-zero exit is a result, not an error."
|
||||||
|
),
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"command": {"type": "string"}},
|
||||||
|
"required": ["command"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _truncate(text: str) -> str:
|
||||||
|
if len(text) <= MAX_OUTPUT_CHARS:
|
||||||
|
return text
|
||||||
|
dropped = len(text) - MAX_OUTPUT_CHARS
|
||||||
|
return f"{text[:MAX_OUTPUT_CHARS]}\n[... {dropped} characters truncated]"
|
||||||
|
|
||||||
|
|
||||||
|
def _child_env() -> dict[str, str]:
|
||||||
|
"""The project's build and test code is arbitrary; it has no business
|
||||||
|
reading the agent's LiteLLM, Matrix, or Gitea credentials."""
|
||||||
|
return {k: v for k, v in os.environ.items() if not k.startswith("NISSE_")}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ToolOutcome:
|
||||||
|
result: str
|
||||||
|
success: bool
|
||||||
|
|
||||||
|
|
||||||
|
class Workspace:
|
||||||
|
def __init__(self, root: Path, *, timeout_s: int = DEFAULT_TIMEOUT_S) -> None:
|
||||||
|
self.root = Path(root)
|
||||||
|
self.root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._timeout_s = timeout_s
|
||||||
|
|
||||||
|
def _resolve(self, path: str) -> Path:
|
||||||
|
candidate = Path(path)
|
||||||
|
return candidate if candidate.is_absolute() else self.root / candidate
|
||||||
|
|
||||||
|
def _inside(self, target: Path) -> bool:
|
||||||
|
try:
|
||||||
|
return target.resolve().is_relative_to(self.root.resolve())
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def dispatch(self, tool: str, arguments: dict[str, Any]) -> ToolOutcome:
|
||||||
|
try:
|
||||||
|
match tool:
|
||||||
|
case "read_file":
|
||||||
|
return self.read_file(str(arguments["path"]))
|
||||||
|
case "write_file":
|
||||||
|
return self.write_file(
|
||||||
|
str(arguments["path"]), str(arguments["content"])
|
||||||
|
)
|
||||||
|
case "list_dir":
|
||||||
|
return self.list_dir(str(arguments.get("path", ".")))
|
||||||
|
case "run_command":
|
||||||
|
return self.run_command(str(arguments["command"]))
|
||||||
|
case _:
|
||||||
|
return ToolOutcome(f"no such tool: {tool}", success=False)
|
||||||
|
except KeyError as exc:
|
||||||
|
return ToolOutcome(f"missing argument {exc} for {tool}", success=False)
|
||||||
|
|
||||||
|
def read_file(self, path: str) -> ToolOutcome:
|
||||||
|
target = self._resolve(path)
|
||||||
|
try:
|
||||||
|
return ToolOutcome(_truncate(target.read_text()), success=True)
|
||||||
|
except (OSError, UnicodeDecodeError) as exc:
|
||||||
|
return ToolOutcome(f"could not read {path}: {exc}", success=False)
|
||||||
|
|
||||||
|
def write_file(self, path: str, content: str) -> ToolOutcome:
|
||||||
|
target = self._resolve(path)
|
||||||
|
# Not a sandbox — the VM is the trust boundary (B3). This is about the
|
||||||
|
# work existing at all: a file written outside the repo is invisible
|
||||||
|
# to git, so the milestone would publish an empty PR while the model
|
||||||
|
# truthfully reports success. Models do guess plausible absolute paths
|
||||||
|
# like /workspace, so say where the workspace actually is.
|
||||||
|
if not self._inside(target):
|
||||||
|
return ToolOutcome(
|
||||||
|
f"refusing to write outside the workspace: {path}. "
|
||||||
|
f"The workspace root is {self.root}; write there, using "
|
||||||
|
"relative paths.",
|
||||||
|
success=False,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text(content)
|
||||||
|
except OSError as exc:
|
||||||
|
return ToolOutcome(f"could not write {path}: {exc}", success=False)
|
||||||
|
return ToolOutcome(f"wrote {len(content)} characters to {path}", success=True)
|
||||||
|
|
||||||
|
def list_dir(self, path: str) -> ToolOutcome:
|
||||||
|
target = self._resolve(path)
|
||||||
|
try:
|
||||||
|
entries = sorted(
|
||||||
|
f"{p.name}/" if p.is_dir() else p.name for p in target.iterdir()
|
||||||
|
)
|
||||||
|
except OSError as exc:
|
||||||
|
return ToolOutcome(f"could not list {path}: {exc}", success=False)
|
||||||
|
return ToolOutcome(_truncate("\n".join(entries) or "(empty)"), success=True)
|
||||||
|
|
||||||
|
def describe(self) -> str:
|
||||||
|
"""Told to the model, so it does not have to guess where it is."""
|
||||||
|
return (
|
||||||
|
f"Your workspace is {self.root}. It is a git repository and the "
|
||||||
|
"only place your work is recorded; commands run there by default. "
|
||||||
|
"Use relative paths."
|
||||||
|
)
|
||||||
|
|
||||||
|
def run_command(self, command: str) -> ToolOutcome:
|
||||||
|
try:
|
||||||
|
# Popen rather than run(): on timeout, run() kills only the direct
|
||||||
|
# child, and TimeoutExpired carries no pid, so a backgrounded
|
||||||
|
# grandchild (a dev server, say) would survive. Holding the pid
|
||||||
|
# lets us kill the whole session.
|
||||||
|
process = subprocess.Popen(
|
||||||
|
command,
|
||||||
|
shell=True,
|
||||||
|
cwd=self.root,
|
||||||
|
env=_child_env(),
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE,
|
||||||
|
stdin=subprocess.DEVNULL,
|
||||||
|
text=True,
|
||||||
|
start_new_session=True,
|
||||||
|
)
|
||||||
|
except OSError as exc:
|
||||||
|
return ToolOutcome(f"could not run command: {exc}", success=False)
|
||||||
|
try:
|
||||||
|
stdout, stderr = process.communicate(timeout=self._timeout_s)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
self._kill_group(process)
|
||||||
|
return ToolOutcome(
|
||||||
|
f"command timed out after {self._timeout_s}s: {command}",
|
||||||
|
success=False,
|
||||||
|
)
|
||||||
|
body = _truncate(
|
||||||
|
f"exit code: {process.returncode}\nstdout:\n{stdout}\nstderr:\n{stderr}"
|
||||||
|
)
|
||||||
|
return ToolOutcome(body, success=process.returncode == 0)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _kill_group(process: "subprocess.Popen[str]") -> None:
|
||||||
|
try:
|
||||||
|
os.killpg(os.getpgid(process.pid), signal.SIGKILL)
|
||||||
|
except (ProcessLookupError, PermissionError):
|
||||||
|
process.kill()
|
||||||
|
# Drain the pipes of the now-dead group so no zombie is left behind.
|
||||||
|
try:
|
||||||
|
process.communicate(timeout=5)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def reset(self) -> None:
|
||||||
|
"""Discard workspace contents; used by tests, not by the agent."""
|
||||||
|
shutil.rmtree(self.root, ignore_errors=True)
|
||||||
|
self.root.mkdir(parents=True, exist_ok=True)
|
||||||
@@ -0,0 +1,210 @@
|
|||||||
|
"""Plan externalisation to Gitea and approvals arriving on either channel
|
||||||
|
(spec A4, Part H)."""
|
||||||
|
|
||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from nisse_dev.approvals import PlanBoard
|
||||||
|
from nisse_dev.canonical import CanonicalEvent, ToolCall
|
||||||
|
from nisse_dev.control import (
|
||||||
|
AWAIT_ARCH_SIGNOFF,
|
||||||
|
AWAIT_MILESTONE_SIGNOFF,
|
||||||
|
MILESTONE_PLANNING,
|
||||||
|
Orchestrator,
|
||||||
|
)
|
||||||
|
from nisse_dev.gitea import Comment, GiteaError
|
||||||
|
from nisse_dev.llm import RoleMap
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.planner import LLMPlanningAgent
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
|
ROLES = RoleMap(executor="x", planner="y", delegate="y", consultant="y")
|
||||||
|
|
||||||
|
|
||||||
|
class FakeGitea:
|
||||||
|
def __init__(self, *, fail: bool = False) -> None:
|
||||||
|
self.issues: list[dict[str, str]] = []
|
||||||
|
self.comments: list[tuple[int, str]] = []
|
||||||
|
self._fail = fail
|
||||||
|
self._next_id = 100
|
||||||
|
|
||||||
|
def create_issue(self, repo: str, title: str, body: str) -> int:
|
||||||
|
if self._fail:
|
||||||
|
raise GiteaError("gitea is down")
|
||||||
|
self.issues.append({"title": title, "body": body})
|
||||||
|
return len(self.issues)
|
||||||
|
|
||||||
|
def post_comment(self, repo: str, index: int, body: str) -> Comment:
|
||||||
|
if self._fail:
|
||||||
|
raise GiteaError("gitea is down")
|
||||||
|
self.comments.append((index, body))
|
||||||
|
self._next_id += 1
|
||||||
|
return Comment(
|
||||||
|
id=self._next_id, author="nisse-dev-testing", body=body, created_at="t"
|
||||||
|
)
|
||||||
|
|
||||||
|
def list_comments(self, repo: str, index: int, since: str | None = None):
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.calls = 0
|
||||||
|
|
||||||
|
def complete(self, events, tools, model, system=None) -> list[CanonicalEvent]:
|
||||||
|
self.calls += 1
|
||||||
|
return [
|
||||||
|
ToolCall(
|
||||||
|
call_id=f"c{self.calls}",
|
||||||
|
tool="submit_plan",
|
||||||
|
arguments={"type": "x", "content": f"plan {self.calls}"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def build(
|
||||||
|
tmp_path: Path, gitea: Any
|
||||||
|
) -> tuple[Orchestrator, StateStore, InMemoryTransport, PlanBoard]:
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
board = PlanBoard(gitea, "o/sandbox")
|
||||||
|
orch = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(ScriptedClient(), store, ROLES),
|
||||||
|
operators={"don"},
|
||||||
|
roles=ROLES,
|
||||||
|
gitea_operators={"dongar"},
|
||||||
|
board=board,
|
||||||
|
)
|
||||||
|
orch.start()
|
||||||
|
return orch, store, transport, board
|
||||||
|
|
||||||
|
|
||||||
|
def comment(body: str, author: str = "dongar", cid: int = 900) -> Comment:
|
||||||
|
return Comment(id=cid, author=author, body=body, created_at="t")
|
||||||
|
|
||||||
|
|
||||||
|
def test_project_start_opens_an_issue_and_posts_the_plan(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
_, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
|
||||||
|
assert gitea.issues[0]["title"] == "nisse-dev: demo"
|
||||||
|
issue, body = gitea.comments[0]
|
||||||
|
assert issue == 1
|
||||||
|
assert "plan 1" in body
|
||||||
|
assert "approve rev-1" in body # the tag is printed for copy-paste (A4)
|
||||||
|
# The store keeps a pointer, not the plan record itself (Part H).
|
||||||
|
assert store.get_state().data["board"]["issue"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_approval_by_gitea_comment_satisfies_the_gate(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, _, _ = build(tmp_path, gitea)
|
||||||
|
orch._transport.receive("don", "use demo")
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1"))
|
||||||
|
|
||||||
|
assert store.get_state().state == AWAIT_MILESTONE_SIGNOFF
|
||||||
|
|
||||||
|
|
||||||
|
def test_stale_revision_from_gitea_is_refused(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "too vague") # now at rev-2
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1"))
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF # not advanced
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-2", cid=901))
|
||||||
|
assert store.get_state().state == AWAIT_MILESTONE_SIGNOFF
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_operator_gitea_comment_cannot_approve(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1", author="drive-by"))
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
|
||||||
|
|
||||||
|
def test_agents_own_comment_cannot_approve_its_plan(tmp_path: Path) -> None:
|
||||||
|
# The agent posts plans as nisse-dev-testing; that login is not on the
|
||||||
|
# operator allowlist, so its own comments can never satisfy its gate (A4).
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1", author="nisse-dev-testing"))
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
|
||||||
|
|
||||||
|
def test_rejection_by_gitea_comment_replans(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("needs more detail on the schema"))
|
||||||
|
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
assert "plan 2" in transport.outbound[-1].text # replanned
|
||||||
|
|
||||||
|
|
||||||
|
def test_matrix_approval_is_echoed_to_gitea(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
_, _, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
|
||||||
|
echoes = [body for _, body in gitea.comments if "approved by don" in body]
|
||||||
|
assert echoes, "the Gitea record must not silently disagree with Matrix"
|
||||||
|
|
||||||
|
|
||||||
|
def test_gitea_approval_is_echoed_to_matrix(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, _, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1"))
|
||||||
|
|
||||||
|
assert any("approved by dongar via gitea" in m.text for m in transport.outbound)
|
||||||
|
|
||||||
|
|
||||||
|
def test_gitea_outage_leaves_matrix_a_complete_channel(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea(fail=True)
|
||||||
|
_, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
|
||||||
|
# Planning proceeded and the gate is usable despite Gitea being down.
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
assert any("Matrix-only" in m.text for m in transport.outbound)
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
assert store.get_state().state == AWAIT_MILESTONE_SIGNOFF
|
||||||
|
|
||||||
|
|
||||||
|
def test_comments_are_ignored_outside_a_gate(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
orch, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve") # arch approved; now at milestone gate
|
||||||
|
transport.receive("don", "approve") # milestone approved, awaiting start
|
||||||
|
before = store.get_state().state
|
||||||
|
|
||||||
|
orch.handle_plan_comment(comment("approve rev-1", cid=902))
|
||||||
|
assert store.get_state().state == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_board_records_the_milestone_plan_too(tmp_path: Path) -> None:
|
||||||
|
gitea = FakeGitea()
|
||||||
|
_, store, transport, _ = build(tmp_path, gitea)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
|
||||||
|
assert store.get_state().state in (MILESTONE_PLANNING, AWAIT_MILESTONE_SIGNOFF)
|
||||||
|
posted = [body for _, body in gitea.comments if "milestone-1" in body]
|
||||||
|
assert posted, "the milestone plan must be externalised as well"
|
||||||
@@ -0,0 +1,304 @@
|
|||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import (
|
||||||
|
AssistantText,
|
||||||
|
CanonicalEvent,
|
||||||
|
DelegationCompleted,
|
||||||
|
DelegationStarted,
|
||||||
|
StoredEvent,
|
||||||
|
ToolCall,
|
||||||
|
)
|
||||||
|
from nisse_dev.delegation import (
|
||||||
|
DELEGATE_TOOLS,
|
||||||
|
Caps,
|
||||||
|
Delegation,
|
||||||
|
)
|
||||||
|
from nisse_dev.git import GitRepo
|
||||||
|
from nisse_dev.llm import LLMError
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import Workspace
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
def __init__(self, turns: list[list[CanonicalEvent]]) -> None:
|
||||||
|
self._turns = turns
|
||||||
|
self.calls: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
events: list[StoredEvent],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
system: str | None = None,
|
||||||
|
) -> list[CanonicalEvent]:
|
||||||
|
self.calls.append({"tools": tools, "model": model, "system": system})
|
||||||
|
if not self._turns:
|
||||||
|
return [AssistantText(text="still thinking", model=model)]
|
||||||
|
return self._turns.pop(0)
|
||||||
|
|
||||||
|
|
||||||
|
class FailingClient:
|
||||||
|
def complete(self, *args: Any, **kwargs: Any) -> list[CanonicalEvent]:
|
||||||
|
raise LLMError("frontier unavailable")
|
||||||
|
|
||||||
|
|
||||||
|
def call(tool: str, cid: str = "c1", **arguments: Any) -> ToolCall:
|
||||||
|
return ToolCall(call_id=cid, tool=tool, arguments=arguments)
|
||||||
|
|
||||||
|
|
||||||
|
def hand_back(succeeded: bool = True, cid: str = "hb", **overrides: Any) -> ToolCall:
|
||||||
|
args = {
|
||||||
|
"changed": "added greet.py",
|
||||||
|
"now_true": "greet() returns a greeting and its test passes",
|
||||||
|
"next": "open the PR",
|
||||||
|
"succeeded": succeeded,
|
||||||
|
}
|
||||||
|
args.update(overrides)
|
||||||
|
return ToolCall(call_id=cid, tool="hand_back", arguments=args)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path: Path) -> StateStore:
|
||||||
|
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def repo(tmp_path: Path) -> GitRepo:
|
||||||
|
r = GitRepo(tmp_path / "work")
|
||||||
|
r.init()
|
||||||
|
r.configure_identity("bot", "bot@example.test")
|
||||||
|
(r.root / "seed.txt").write_text("seed")
|
||||||
|
r.commit_all("seed")
|
||||||
|
return r
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def ws(repo: GitRepo) -> Workspace:
|
||||||
|
return Workspace(repo.root, timeout_s=10)
|
||||||
|
|
||||||
|
|
||||||
|
def build(client: Any, store: StateStore, ws: Workspace, repo: GitRepo, **kw: Any):
|
||||||
|
return Delegation(client, store, ws, repo, model="claude-haiku-4-5", **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def test_successful_delegation_returns_a_structured_handback(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("write_file", path="greet.py", content="def greet(): ...")],
|
||||||
|
[hand_back()],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="add greet", branch="d1")
|
||||||
|
|
||||||
|
assert result.usable
|
||||||
|
assert result.handback is not None
|
||||||
|
assert "greet.py" in result.handback.changed
|
||||||
|
assert "what to do next" in result.handback.render().lower()
|
||||||
|
assert result.artifacts == ["greet.py"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_delegation_works_on_its_own_branch(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[[call("write_file", path="a.py", content="x")], [hand_back()]]
|
||||||
|
)
|
||||||
|
build(client, store, ws, repo).run(brief="b", branch="deleg-1")
|
||||||
|
assert repo.current_branch() == "deleg-1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_abort_discards_the_branch_and_its_work(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
# A13: abort must leave nothing to resume on top of.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("write_file", path="half-done.py", content="oops")],
|
||||||
|
[hand_back(succeeded=False, changed="could not finish")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="b", branch="deleg-2")
|
||||||
|
|
||||||
|
assert not result.usable
|
||||||
|
assert repo.current_branch() == "main"
|
||||||
|
assert not (repo.root / "half-done.py").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_turn_cap_is_hard(store: StateStore, ws: Workspace, repo: GitRepo) -> None:
|
||||||
|
forever = [[call("list_dir", path=".")] for _ in range(20)]
|
||||||
|
client = ScriptedClient(forever)
|
||||||
|
result = build(client, store, ws, repo, caps=Caps(turns=3)).run(
|
||||||
|
brief="b", branch="d"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result.status == "capped"
|
||||||
|
assert "turn cap of 3" in result.note
|
||||||
|
assert len(client.calls) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_wall_clock_cap_is_hard(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
ticks = iter([0.0, 1.0, 99.0, 99.0, 99.0])
|
||||||
|
client = ScriptedClient([[call("list_dir", path=".")] for _ in range(10)])
|
||||||
|
result = build(
|
||||||
|
client,
|
||||||
|
store,
|
||||||
|
ws,
|
||||||
|
repo,
|
||||||
|
caps=Caps(turns=10, wall_clock_s=60.0),
|
||||||
|
now=lambda: next(ticks),
|
||||||
|
).run(brief="b", branch="d")
|
||||||
|
|
||||||
|
assert result.status == "capped"
|
||||||
|
assert "wall-clock cap" in result.note
|
||||||
|
|
||||||
|
|
||||||
|
def test_cap_breach_is_not_usable_and_discards_work(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient([[call("write_file", path="partial.py", content="x")]] * 5)
|
||||||
|
result = build(client, store, ws, repo, caps=Caps(turns=2)).run(
|
||||||
|
brief="b", branch="d"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert not result.usable # the caller must gate, not resume (A13)
|
||||||
|
assert not (repo.root / "partial.py").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_events_are_checkpointed_as_they_arrive(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[[call("write_file", path="a.py", content="x")], [hand_back()]]
|
||||||
|
)
|
||||||
|
build(client, store, ws, repo).run(brief="add a", branch="d")
|
||||||
|
|
||||||
|
kinds = [type(e.event).__name__ for e in store.read_events()]
|
||||||
|
assert kinds == [
|
||||||
|
"DelegationStarted",
|
||||||
|
"ToolCall",
|
||||||
|
"ToolResult",
|
||||||
|
"ToolCall", # hand_back
|
||||||
|
"ToolResult",
|
||||||
|
"DelegationCompleted",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_composite_events_record_the_brief_caps_and_outcome(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[[call("write_file", path="a.py", content="x")], [hand_back()]]
|
||||||
|
)
|
||||||
|
caps = Caps(turns=7, tokens=1234, wall_clock_s=42.0)
|
||||||
|
build(client, store, ws, repo, caps=caps).run(brief="the brief", branch="d")
|
||||||
|
|
||||||
|
events = [e.event for e in store.read_events()]
|
||||||
|
started = next(e for e in events if isinstance(e, DelegationStarted))
|
||||||
|
completed = next(e for e in events if isinstance(e, DelegationCompleted))
|
||||||
|
assert started.brief == "the brief"
|
||||||
|
assert started.model == "claude-haiku-4-5"
|
||||||
|
assert started.caps == {"turns": 7, "tokens": 1234, "wall_clock_s": 42.0}
|
||||||
|
# A16: this pair is the eval record, so it must carry the outcome.
|
||||||
|
assert completed.exit_status == "handback"
|
||||||
|
assert "greet()" in completed.handback
|
||||||
|
|
||||||
|
|
||||||
|
def test_hand_back_tool_is_offered_alongside_the_executor_tools() -> None:
|
||||||
|
names = [t["function"]["name"] for t in DELEGATE_TOOLS]
|
||||||
|
assert names[-1] == "hand_back"
|
||||||
|
assert "run_command" in names # full tool access, not a consult (A13)
|
||||||
|
|
||||||
|
|
||||||
|
def test_malformed_handback_is_rejected_and_the_delegation_continues(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[hand_back(cid="bad", changed=" ")], # blank field
|
||||||
|
[call("write_file", path="a.py", content="x")],
|
||||||
|
[hand_back(cid="good")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="b", branch="d")
|
||||||
|
|
||||||
|
assert result.usable # recovered rather than accepting a useless note
|
||||||
|
assert len(client.calls) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_boolean_success_flag_is_rejected(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("write_file", path="a.py", content="x")],
|
||||||
|
[hand_back(succeeded="yes")], # not a bool: unparseable
|
||||||
|
[hand_back(cid="ok")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="b", branch="d")
|
||||||
|
assert result.usable
|
||||||
|
assert len(client.calls) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_client_failure_is_a_defined_return_not_a_crash(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
result = build(FailingClient(), store, ws, repo).run(brief="b", branch="d")
|
||||||
|
assert result.status == "failed"
|
||||||
|
assert "frontier unavailable" in result.note
|
||||||
|
assert not result.usable
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_delegate_that_never_hands_back_is_capped_not_believed(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
# Prose forever: the delegate never reports, so its tree state is unknown.
|
||||||
|
client = ScriptedClient([])
|
||||||
|
result = build(client, store, ws, repo, caps=Caps(turns=3)).run(
|
||||||
|
brief="b", branch="d"
|
||||||
|
)
|
||||||
|
assert result.status == "capped"
|
||||||
|
assert not result.usable
|
||||||
|
|
||||||
|
|
||||||
|
def test_success_claimed_without_any_change_is_refused(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
# The handback plus file state is the whole interface back, so a note
|
||||||
|
# describing work that was never done is worse than no note.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[hand_back(cid="early")],
|
||||||
|
[call("write_file", path="real.py", content="x")],
|
||||||
|
[hand_back(cid="earned")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="b", branch="d")
|
||||||
|
|
||||||
|
assert result.usable
|
||||||
|
assert len(client.calls) == 3 # the first claim was sent back
|
||||||
|
assert (repo.root / "real.py").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_failure_may_be_reported_without_doing_work(
|
||||||
|
store: StateStore, ws: Workspace, repo: GitRepo
|
||||||
|
) -> None:
|
||||||
|
# "This sub-task is impossible" is a legitimate finding that requires no
|
||||||
|
# changes; only a claim of success needs evidence.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[[hand_back(succeeded=False, changed="nothing: the API does not exist")]]
|
||||||
|
)
|
||||||
|
result = build(client, store, ws, repo).run(brief="b", branch="d")
|
||||||
|
|
||||||
|
assert result.status == "failed"
|
||||||
|
assert not result.usable
|
||||||
|
assert len(client.calls) == 1 # believed immediately, not challenged
|
||||||
@@ -0,0 +1,225 @@
|
|||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import AssistantText, CanonicalEvent, StoredEvent, ToolCall
|
||||||
|
from nisse_dev.executor import LLMExecutor
|
||||||
|
from nisse_dev.llm import LLMError, RoleMap
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import Workspace
|
||||||
|
|
||||||
|
ROLES = RoleMap(
|
||||||
|
executor="llama3.2",
|
||||||
|
planner="claude-haiku-4-5",
|
||||||
|
delegate="claude-haiku-4-5",
|
||||||
|
consultant="claude-haiku-4-5",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
def __init__(self, turns: list[list[CanonicalEvent]]) -> None:
|
||||||
|
self._turns = turns
|
||||||
|
self.calls: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
events: list[StoredEvent],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
system: str | None = None,
|
||||||
|
) -> list[CanonicalEvent]:
|
||||||
|
self.calls.append({"tools": tools, "model": model, "system": system})
|
||||||
|
if not self._turns:
|
||||||
|
return [AssistantText(text="done", model=model)]
|
||||||
|
return self._turns.pop(0)
|
||||||
|
|
||||||
|
|
||||||
|
class FailingClient:
|
||||||
|
def complete(self, *args: Any, **kwargs: Any) -> list[CanonicalEvent]:
|
||||||
|
raise LLMError("proxy exploded")
|
||||||
|
|
||||||
|
|
||||||
|
def call(tool: str, call_id: str = "c1", **arguments: Any) -> ToolCall:
|
||||||
|
return ToolCall(call_id=call_id, tool=tool, arguments=arguments)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path: Path) -> StateStore:
|
||||||
|
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def ws(tmp_path: Path) -> Workspace:
|
||||||
|
return Workspace(tmp_path / "work", timeout_s=10)
|
||||||
|
|
||||||
|
|
||||||
|
def build(client: Any, store: StateStore, ws: Workspace, **kw: Any) -> LLMExecutor:
|
||||||
|
return LLMExecutor(client, store, ROLES, ws, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_calls_are_executed_and_results_logged(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("write_file", path="app.py", content="print(1)\n")],
|
||||||
|
[AssistantText(text="wrote the file", model="llama3.2")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
outcome = build(client, store, ws).run(milestone=1)
|
||||||
|
|
||||||
|
assert outcome.status == "completed"
|
||||||
|
assert (ws.root / "app.py").read_text() == "print(1)\n"
|
||||||
|
kinds = [type(e.event).__name__ for e in store.read_events()]
|
||||||
|
assert kinds == ["ToolCall", "ToolResult", "AssistantText"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_uses_executor_role_and_reduced_manifest(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient([])
|
||||||
|
build(client, store, ws).run(milestone=2)
|
||||||
|
assert client.calls[0]["model"] == "llama3.2" # never the planner
|
||||||
|
names = [t["function"]["name"] for t in client.calls[0]["tools"]]
|
||||||
|
assert names == ["read_file", "write_file", "list_dir", "run_command"]
|
||||||
|
assert "milestone 2" in client.calls[0]["system"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_tool_call_after_work_ends_the_milestone(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("write_file", path="a.py", content="x")],
|
||||||
|
[AssistantText(text="all steps done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert build(client, store, ws).run(milestone=1).status == "completed"
|
||||||
|
|
||||||
|
|
||||||
|
def test_completion_claimed_without_any_work_is_challenged(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
# A model that answers with prose on turn one has done nothing; believing
|
||||||
|
# it would record an empty milestone as a success.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[AssistantText(text="looks done to me", model="m")],
|
||||||
|
[call("write_file", path="a.py", content="x")],
|
||||||
|
[AssistantText(text="now it is done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
outcome = build(client, store, ws).run(milestone=1)
|
||||||
|
assert outcome.status == "completed"
|
||||||
|
assert (ws.root / "a.py").exists() # the nudge produced actual work
|
||||||
|
assert "not made any change yet" in client.calls[1]["system"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_persisting_in_an_empty_claim_escalates(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
prose = [AssistantText(text="nothing to do", model="m")]
|
||||||
|
client = ScriptedClient([prose, prose])
|
||||||
|
outcome = build(client, store, ws).run(milestone=1)
|
||||||
|
assert outcome.status == "escalate"
|
||||||
|
assert "without making any change" in outcome.note
|
||||||
|
|
||||||
|
|
||||||
|
def test_two_consecutive_command_failures_escalate(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("run_command", command="exit 1")],
|
||||||
|
[call("run_command", command="exit 1")],
|
||||||
|
[call("run_command", command="exit 1")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
outcome = build(client, store, ws, failure_limit=2).run(milestone=1)
|
||||||
|
assert outcome.status == "escalate"
|
||||||
|
assert "2 consecutive command failures" in outcome.note
|
||||||
|
assert len(client.calls) == 2 # stopped, did not keep trying
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_success_resets_the_failure_count(store: StateStore, ws: Workspace) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("run_command", command="exit 1")],
|
||||||
|
[call("run_command", command="true")],
|
||||||
|
[call("run_command", command="exit 1")],
|
||||||
|
[AssistantText(text="done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert build(client, store, ws, failure_limit=2).run(milestone=1).status == (
|
||||||
|
"completed"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_failed_reads_do_not_count_toward_escalation(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
# Two failed reads in a row must not trip the failure limit: looking for
|
||||||
|
# a file that is not there is the model orienting itself, not failing.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("read_file", path="missing-a.txt")],
|
||||||
|
[call("read_file", path="missing-b.txt")],
|
||||||
|
[call("write_file", path="a.py", content="x")],
|
||||||
|
[AssistantText(text="done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert build(client, store, ws, failure_limit=2).run(milestone=1).status == (
|
||||||
|
"completed"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_reading_alone_is_not_doing_the_work(store: StateStore, ws: Workspace) -> None:
|
||||||
|
# Exploration is not evidence a milestone was carried out, so a
|
||||||
|
# completion claim backed only by reads is still challenged.
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[call("list_dir", path=".")],
|
||||||
|
[AssistantText(text="looks done to me", model="m")],
|
||||||
|
[AssistantText(text="really, done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
outcome = build(client, store, ws).run(milestone=1)
|
||||||
|
assert outcome.status == "escalate"
|
||||||
|
|
||||||
|
|
||||||
|
def test_turn_cap_is_enforced(store: StateStore, ws: Workspace) -> None:
|
||||||
|
forever = [[call("list_dir", path=".")] for _ in range(50)]
|
||||||
|
client = ScriptedClient(forever)
|
||||||
|
outcome = build(client, store, ws, max_turns=3).run(milestone=1)
|
||||||
|
assert outcome.status == "capped"
|
||||||
|
assert "3-turn cap" in outcome.note
|
||||||
|
assert len(client.calls) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_llm_error_escalates_rather_than_crashing(
|
||||||
|
store: StateStore, ws: Workspace
|
||||||
|
) -> None:
|
||||||
|
outcome = build(FailingClient(), store, ws).run(milestone=1)
|
||||||
|
assert outcome.status == "escalate"
|
||||||
|
assert "proxy exploded" in outcome.note
|
||||||
|
|
||||||
|
|
||||||
|
def test_multiple_calls_in_one_turn_all_run(store: StateStore, ws: Workspace) -> None:
|
||||||
|
client = ScriptedClient(
|
||||||
|
[
|
||||||
|
[
|
||||||
|
call("write_file", call_id="a", path="one.txt", content="1"),
|
||||||
|
call("write_file", call_id="b", path="two.txt", content="2"),
|
||||||
|
],
|
||||||
|
[AssistantText(text="done", model="m")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
build(client, store, ws).run(milestone=1)
|
||||||
|
assert (ws.root / "one.txt").exists() and (ws.root / "two.txt").exists()
|
||||||
|
results = [
|
||||||
|
e.event for e in store.read_events() if type(e.event).__name__ == "ToolResult"
|
||||||
|
]
|
||||||
|
assert {r.call_id for r in results} == {"a", "b"}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.git import GitError, GitRepo, _scrub, authenticated_url
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def repo(tmp_path: Path) -> GitRepo:
|
||||||
|
r = GitRepo(tmp_path / "work")
|
||||||
|
r.init()
|
||||||
|
r.configure_identity("nisse-dev-testing", "nisse@example.test")
|
||||||
|
return r
|
||||||
|
|
||||||
|
|
||||||
|
def test_init_creates_repo_on_default_branch(repo: GitRepo) -> None:
|
||||||
|
assert repo.current_branch() == "main"
|
||||||
|
|
||||||
|
|
||||||
|
def test_commit_all_returns_sha_and_branch_switching(repo: GitRepo) -> None:
|
||||||
|
(repo.root / "a.txt").write_text("a")
|
||||||
|
sha = repo.commit_all("first")
|
||||||
|
assert sha is not None
|
||||||
|
assert sha == repo.head_sha()
|
||||||
|
|
||||||
|
repo.checkout_new_branch("milestone-1")
|
||||||
|
assert repo.current_branch() == "milestone-1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_commit_with_nothing_staged_returns_none(repo: GitRepo) -> None:
|
||||||
|
(repo.root / "a.txt").write_text("a")
|
||||||
|
repo.commit_all("first")
|
||||||
|
# An empty milestone is a fact to report, not an error to raise.
|
||||||
|
assert repo.commit_all("nothing changed") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_has_changes_tracks_the_worktree(repo: GitRepo) -> None:
|
||||||
|
assert not repo.has_changes()
|
||||||
|
(repo.root / "a.txt").write_text("a")
|
||||||
|
assert repo.has_changes()
|
||||||
|
|
||||||
|
|
||||||
|
def test_authenticated_url_shape() -> None:
|
||||||
|
url = authenticated_url("https://gitea.test/", "o/r", "bot", "secret")
|
||||||
|
assert url == "https://bot:secret@gitea.test/o/r.git"
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_failure_does_not_leak_the_token(repo: GitRepo) -> None:
|
||||||
|
(repo.root / "a.txt").write_text("a")
|
||||||
|
repo.commit_all("first")
|
||||||
|
url = authenticated_url("https://127.0.0.1:1", "o/r", "bot", "sup3rsecret")
|
||||||
|
with pytest.raises(GitError) as excinfo:
|
||||||
|
repo.push(url, "main", token="sup3rsecret")
|
||||||
|
# git redacts credentials from its own messages, but that is git's
|
||||||
|
# behaviour, not ours; the scrub below is the guarantee we control.
|
||||||
|
assert "sup3rsecret" not in str(excinfo.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_scrub_removes_the_token_when_git_does_echo_it() -> None:
|
||||||
|
assert _scrub("failed to push to https://bot:s3cret@h/o/r.git", "s3cret") == (
|
||||||
|
"failed to push to https://bot:***@h/o/r.git"
|
||||||
|
)
|
||||||
|
assert _scrub("no secret here", None) == "no secret here"
|
||||||
|
|
||||||
|
|
||||||
|
def test_git_error_on_bad_command(tmp_path: Path) -> None:
|
||||||
|
repo = GitRepo(tmp_path / "not-a-repo")
|
||||||
|
repo.root.mkdir()
|
||||||
|
with pytest.raises(GitError):
|
||||||
|
repo.head_sha()
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
import json
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.gitea import (
|
||||||
|
PR_PAGE_SIZE,
|
||||||
|
Comment,
|
||||||
|
GiteaClient,
|
||||||
|
GiteaError,
|
||||||
|
pull_request_reconciler,
|
||||||
|
)
|
||||||
|
from nisse_dev.state import Intent
|
||||||
|
|
||||||
|
INTENT = Intent(
|
||||||
|
id=1,
|
||||||
|
kind="open_pr",
|
||||||
|
payload={"branch": "milestone-1"},
|
||||||
|
status="pending",
|
||||||
|
created_at="t",
|
||||||
|
resolved_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def client(handler) -> GiteaClient:
|
||||||
|
return GiteaClient(
|
||||||
|
"https://gitea.test/", "tok", transport=httpx.MockTransport(handler)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def pr_json(number: int, head: str, state: str = "open") -> dict:
|
||||||
|
return {
|
||||||
|
"number": number,
|
||||||
|
"state": state,
|
||||||
|
"head": {"ref": head},
|
||||||
|
"html_url": f"https://gitea.test/o/r/pulls/{number}",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_open_pull_request_posts_expected_body() -> None:
|
||||||
|
seen: dict = {}
|
||||||
|
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
seen["url"] = str(request.url)
|
||||||
|
seen["auth"] = request.headers.get("authorization")
|
||||||
|
seen["body"] = json.loads(request.content)
|
||||||
|
return httpx.Response(201, json=pr_json(7, "milestone-1"))
|
||||||
|
|
||||||
|
pr = client(handler).open_pull_request(
|
||||||
|
"o/r", head="milestone-1", base="main", title="T", body="B"
|
||||||
|
)
|
||||||
|
assert seen["url"] == "https://gitea.test/api/v1/repos/o/r/pulls"
|
||||||
|
assert seen["auth"] == "token tok"
|
||||||
|
assert seen["body"] == {
|
||||||
|
"head": "milestone-1",
|
||||||
|
"base": "main",
|
||||||
|
"title": "T",
|
||||||
|
"body": "B",
|
||||||
|
}
|
||||||
|
assert pr.number == 7
|
||||||
|
assert pr.head == "milestone-1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_pull_request_matches_head_branch() -> None:
|
||||||
|
handler = lambda request: httpx.Response(200, json=[pr_json(3, "other")])
|
||||||
|
assert client(handler).find_pull_request("o/r", "milestone-1") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_pull_request_also_finds_closed() -> None:
|
||||||
|
# A crash after opening must not be answered with "nothing open, open
|
||||||
|
# another" — the effect already happened.
|
||||||
|
handler = lambda request: httpx.Response(
|
||||||
|
200, json=[pr_json(4, "milestone-1", "closed")]
|
||||||
|
)
|
||||||
|
found = client(handler).find_pull_request("o/r", "milestone-1")
|
||||||
|
assert found is not None
|
||||||
|
assert found.number == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_pull_request_pages_past_the_first_page() -> None:
|
||||||
|
# The match sits on page 2; sampling only the newest page would report
|
||||||
|
# "absent" and licence a duplicate PR.
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
page = int(request.url.params.get("page", 1))
|
||||||
|
if page == 1:
|
||||||
|
return httpx.Response(
|
||||||
|
200, json=[pr_json(i, f"other-{i}") for i in range(PR_PAGE_SIZE)]
|
||||||
|
)
|
||||||
|
return httpx.Response(200, json=[pr_json(999, "milestone-1", "closed")])
|
||||||
|
|
||||||
|
found = client(handler).find_pull_request("o/r", "milestone-1")
|
||||||
|
assert found is not None
|
||||||
|
assert found.number == 999
|
||||||
|
|
||||||
|
|
||||||
|
def test_short_page_ends_the_search_definitively() -> None:
|
||||||
|
calls: list[int] = []
|
||||||
|
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
calls.append(int(request.url.params.get("page", 1)))
|
||||||
|
return httpx.Response(200, json=[pr_json(1, "other")])
|
||||||
|
|
||||||
|
assert client(handler).find_pull_request("o/r", "milestone-1") is None
|
||||||
|
assert calls == [1] # one short page proves absence; no further paging
|
||||||
|
|
||||||
|
|
||||||
|
def test_unbounded_history_raises_rather_than_guessing() -> None:
|
||||||
|
# Never answer "absent" from an incomplete search: the reconciler turns
|
||||||
|
# this error into "unverifiable" and gates to the human.
|
||||||
|
handler = lambda request: httpx.Response(
|
||||||
|
200, json=[pr_json(i, f"other-{i}") for i in range(PR_PAGE_SIZE)]
|
||||||
|
)
|
||||||
|
with pytest.raises(GiteaError, match="without reaching the end"):
|
||||||
|
client(handler).find_pull_request("o/r", "milestone-1")
|
||||||
|
|
||||||
|
|
||||||
|
def test_error_status_raises_gitea_error() -> None:
|
||||||
|
handler = lambda request: httpx.Response(403, text="forbidden")
|
||||||
|
with pytest.raises(GiteaError, match="403"):
|
||||||
|
client(handler).whoami()
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_and_post_comments() -> None:
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
if request.method == "POST":
|
||||||
|
return httpx.Response(
|
||||||
|
201,
|
||||||
|
json={
|
||||||
|
"id": 9,
|
||||||
|
"user": {"login": "don"},
|
||||||
|
"body": "hi",
|
||||||
|
"created_at": "t",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return httpx.Response(
|
||||||
|
200,
|
||||||
|
json=[
|
||||||
|
{
|
||||||
|
"id": 8,
|
||||||
|
"user": {"login": "don"},
|
||||||
|
"body": "approve rev-1",
|
||||||
|
"created_at": "t",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
c = client(handler)
|
||||||
|
assert c.list_comments("o/r", 1) == [
|
||||||
|
Comment(id=8, author="don", body="approve rev-1", created_at="t")
|
||||||
|
]
|
||||||
|
assert c.post_comment("o/r", 1, "hi").id == 9
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_comments_passes_since() -> None:
|
||||||
|
seen: dict = {}
|
||||||
|
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
seen["since"] = request.url.params.get("since")
|
||||||
|
return httpx.Response(200, json=[])
|
||||||
|
|
||||||
|
client(handler).list_comments("o/r", 1, since="2026-09-07T00:00:00Z")
|
||||||
|
assert seen["since"] == "2026-09-07T00:00:00Z"
|
||||||
|
|
||||||
|
|
||||||
|
def test_from_env_requires_config() -> None:
|
||||||
|
with pytest.raises(GiteaError, match="NISSE_GITEA_URL"):
|
||||||
|
GiteaClient.from_env({})
|
||||||
|
|
||||||
|
|
||||||
|
# --- reconciler (A3 external truth) --------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_reconciler_reports_existing_pr_as_done() -> None:
|
||||||
|
handler = lambda request: httpx.Response(200, json=[pr_json(5, "milestone-1")])
|
||||||
|
assert pull_request_reconciler(client(handler), "o/r")(INTENT) is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_reconciler_reports_missing_pr_as_not_done() -> None:
|
||||||
|
handler = lambda request: httpx.Response(200, json=[])
|
||||||
|
assert pull_request_reconciler(client(handler), "o/r")(INTENT) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_unreachable_gitea_is_unverifiable_not_a_guess() -> None:
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
raise httpx.ConnectError("refused")
|
||||||
|
|
||||||
|
# None gates to the human: a delayed decision is cheap, a duplicate PR
|
||||||
|
# is not.
|
||||||
|
assert pull_request_reconciler(client(handler), "o/r")(INTENT) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_intent_without_a_branch_is_unverifiable() -> None:
|
||||||
|
handler = lambda request: httpx.Response(200, json=[])
|
||||||
|
blank = Intent(
|
||||||
|
id=2,
|
||||||
|
kind="open_pr",
|
||||||
|
payload={},
|
||||||
|
status="pending",
|
||||||
|
created_at="t",
|
||||||
|
resolved_at=None,
|
||||||
|
)
|
||||||
|
assert pull_request_reconciler(client(handler), "o/r")(blank) is None
|
||||||
@@ -11,16 +11,17 @@ from nisse_dev.canonical import (
|
|||||||
UserMessage,
|
UserMessage,
|
||||||
)
|
)
|
||||||
from nisse_dev.envfile import load_env_file
|
from nisse_dev.envfile import load_env_file
|
||||||
from nisse_dev.llm import LiteLLMClient, LLMError, RoleMap
|
from nisse_dev.llm import RATE_LIMIT_RETRY_S, LiteLLMClient, LLMError, RoleMap
|
||||||
|
|
||||||
EVENTS = [StoredEvent(id=1, ts="t", event=UserMessage(text="hi", source="m:1"))]
|
EVENTS = [StoredEvent(id=1, ts="t", event=UserMessage(text="hi", source="m:1"))]
|
||||||
|
|
||||||
|
|
||||||
def make_client(handler) -> LiteLLMClient:
|
def make_client(handler, sleep=lambda _s: None) -> LiteLLMClient:
|
||||||
return LiteLLMClient(
|
return LiteLLMClient(
|
||||||
"http://litellm.test/v1",
|
"http://litellm.test/v1",
|
||||||
"sk-test",
|
"sk-test",
|
||||||
transport=httpx.MockTransport(handler),
|
transport=httpx.MockTransport(handler),
|
||||||
|
sleep=sleep, # never wait in unit tests
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -109,9 +110,34 @@ def test_non_object_tool_arguments_raise_llm_error() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def test_http_error_status_raises_llm_error() -> None:
|
def test_http_error_status_raises_llm_error() -> None:
|
||||||
client = make_client(lambda request: httpx.Response(429, text="rate limited"))
|
client = make_client(lambda request: httpx.Response(500, text="boom"))
|
||||||
|
with pytest.raises(LLMError, match="500"):
|
||||||
|
client.complete(EVENTS, tools=None, model="local/qwen")
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_is_retried_once_then_surfaced() -> None:
|
||||||
|
# A 429 is transient. Losing a long delegation to one costs far more
|
||||||
|
# than waiting out the window, but retrying forever would hide a real
|
||||||
|
# ceiling, so it is retried exactly once.
|
||||||
|
attempts: list[int] = []
|
||||||
|
waited: list[float] = []
|
||||||
|
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
attempts.append(1)
|
||||||
|
return httpx.Response(429, text="rate limited")
|
||||||
|
|
||||||
|
client = make_client(handler, sleep=waited.append)
|
||||||
with pytest.raises(LLMError, match="429"):
|
with pytest.raises(LLMError, match="429"):
|
||||||
client.complete(EVENTS, tools=None, model="local/qwen")
|
client.complete(EVENTS, tools=None, model="local/qwen")
|
||||||
|
assert len(attempts) == 2
|
||||||
|
assert waited == [RATE_LIMIT_RETRY_S]
|
||||||
|
|
||||||
|
|
||||||
|
def test_rate_limit_that_clears_succeeds() -> None:
|
||||||
|
responses = [httpx.Response(429, text="slow down"), completion({"content": "hi"})]
|
||||||
|
client = make_client(lambda request: responses.pop(0), sleep=lambda _s: None)
|
||||||
|
[event] = client.complete(EVENTS, tools=None, model="local/qwen")
|
||||||
|
assert isinstance(event, AssistantText)
|
||||||
|
|
||||||
|
|
||||||
def test_connection_failure_raises_llm_error() -> None:
|
def test_connection_failure_raises_llm_error() -> None:
|
||||||
|
|||||||
@@ -12,7 +12,14 @@ from nisse_dev.canonical import (
|
|||||||
ToolCall,
|
ToolCall,
|
||||||
UserMessage,
|
UserMessage,
|
||||||
)
|
)
|
||||||
from nisse_dev.control import AskHuman, Orchestrator, PlanningError, SubmitPlan
|
from nisse_dev.control import (
|
||||||
|
AWAIT_ARCH_SIGNOFF,
|
||||||
|
AWAIT_MILESTONE_SIGNOFF,
|
||||||
|
AskHuman,
|
||||||
|
Orchestrator,
|
||||||
|
PlanningError,
|
||||||
|
SubmitPlan,
|
||||||
|
)
|
||||||
from nisse_dev.llm import RoleMap
|
from nisse_dev.llm import RoleMap
|
||||||
from nisse_dev.oplog import OpLog
|
from nisse_dev.oplog import OpLog
|
||||||
from nisse_dev.planner import PLANNER_TOOLS, LLMPlanningAgent
|
from nisse_dev.planner import PLANNER_TOOLS, LLMPlanningAgent
|
||||||
@@ -108,7 +115,18 @@ def test_model_output_is_appended_to_the_log(store: StateStore) -> None:
|
|||||||
)
|
)
|
||||||
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
||||||
kinds = [type(e.event).__name__ for e in store.read_events()]
|
kinds = [type(e.event).__name__ for e in store.read_events()]
|
||||||
assert kinds == ["AssistantText", "ToolCall"]
|
assert kinds == ["AssistantText", "ToolCall", "ToolResult"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_gate_tool_call_is_answered_in_the_log(store: StateStore) -> None:
|
||||||
|
# submit_plan ends the turn and has no natural result, but a tool call
|
||||||
|
# left dangling makes the next render invalid for the provider.
|
||||||
|
client = ScriptedClient([[plan_call("the plan")]])
|
||||||
|
LLMPlanningAgent(client, store, ROLES).plan("arch", "demo")
|
||||||
|
events = [e.event for e in store.read_events()]
|
||||||
|
assert [type(e).__name__ for e in events] == ["ToolCall", "ToolResult"]
|
||||||
|
assert events[1].call_id == events[0].call_id
|
||||||
|
assert "posted for approval" in events[1].result
|
||||||
|
|
||||||
|
|
||||||
def test_prose_only_turn_retries_once_then_raises(store: StateStore) -> None:
|
def test_prose_only_turn_retries_once_then_raises(store: StateStore) -> None:
|
||||||
@@ -170,7 +188,13 @@ def test_use_project_produces_llm_plan_and_transcript(tmp_path: Path) -> None:
|
|||||||
|
|
||||||
events = [e.event for e in store.read_events()]
|
events = [e.event for e in store.read_events()]
|
||||||
kinds = [type(e).__name__ for e in events]
|
kinds = [type(e).__name__ for e in events]
|
||||||
assert kinds == ["UserMessage", "ToolCall", "PlanSubmitted", "GateOpened"]
|
assert kinds == [
|
||||||
|
"UserMessage",
|
||||||
|
"ToolCall",
|
||||||
|
"ToolResult",
|
||||||
|
"PlanSubmitted",
|
||||||
|
"GateOpened",
|
||||||
|
]
|
||||||
submitted = next(e for e in events if isinstance(e, PlanSubmitted))
|
submitted = next(e for e in events if isinstance(e, PlanSubmitted))
|
||||||
assert submitted.model == "anthropic/claude-opus-5" # pinned planner
|
assert submitted.model == "anthropic/claude-opus-5" # pinned planner
|
||||||
|
|
||||||
@@ -195,6 +219,53 @@ def test_planning_failure_gates_to_await_input(tmp_path: Path) -> None:
|
|||||||
assert transport.outbound[-1].kind == "question"
|
assert transport.outbound[-1].kind == "question"
|
||||||
|
|
||||||
|
|
||||||
|
def test_operator_can_recover_from_a_planning_failure(tmp_path: Path) -> None:
|
||||||
|
# Regression: AWAIT_INPUT reached by any path other than ask_human used to
|
||||||
|
# record no pending item, so every operator reply was refused with "no
|
||||||
|
# pending question" and the agent was stuck for good.
|
||||||
|
prose = [AssistantText(text="prose", model="m")]
|
||||||
|
client = ScriptedClient([prose, prose, [plan_call("recovered plan")]])
|
||||||
|
_, store, transport = build(tmp_path, client)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
assert store.get_state().state == "AWAIT_INPUT"
|
||||||
|
|
||||||
|
transport.receive("don", "try again, keep it short")
|
||||||
|
|
||||||
|
assert store.get_state().state == AWAIT_ARCH_SIGNOFF
|
||||||
|
assert "recovered plan" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_operator_can_recover_after_a_stopped_milestone(tmp_path: Path) -> None:
|
||||||
|
class StoppedExecutor:
|
||||||
|
def run(self, milestone: int) -> Any:
|
||||||
|
from nisse_dev.executor import ExecutorOutcome
|
||||||
|
|
||||||
|
return ExecutorOutcome("escalate", "tests failed twice")
|
||||||
|
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
client = ScriptedClient([[plan_call("p1")], [plan_call("p2")]])
|
||||||
|
orch = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(client, store, ROLES),
|
||||||
|
operators={"don"},
|
||||||
|
roles=ROLES,
|
||||||
|
executor=StoppedExecutor(),
|
||||||
|
)
|
||||||
|
orch.start()
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
transport.receive("don", "start milestone 1")
|
||||||
|
assert store.get_state().state == "AWAIT_INPUT"
|
||||||
|
|
||||||
|
transport.receive("don", "ok, I fixed the dependency")
|
||||||
|
|
||||||
|
assert store.get_state().state == AWAIT_MILESTONE_SIGNOFF
|
||||||
|
assert "start milestone 1" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
def test_rejection_feedback_reaches_the_planner_via_the_log(
|
def test_rejection_feedback_reaches_the_planner_via_the_log(
|
||||||
tmp_path: Path,
|
tmp_path: Path,
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
import io
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.gitea import Comment, GiteaError
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.poller import MAX_BACKOFF_S, CommentPoller
|
||||||
|
|
||||||
|
|
||||||
|
class FakeGitea:
|
||||||
|
def __init__(self, pages: list[list[Comment]] | None = None) -> None:
|
||||||
|
self.pages = pages or []
|
||||||
|
self.calls = 0
|
||||||
|
self.fail_times = 0
|
||||||
|
|
||||||
|
def list_comments(self, repo: str, index: int, since: str | None = None):
|
||||||
|
self.calls += 1
|
||||||
|
if self.fail_times > 0:
|
||||||
|
self.fail_times -= 1
|
||||||
|
raise GiteaError("gitea unreachable")
|
||||||
|
if not self.pages:
|
||||||
|
return []
|
||||||
|
return self.pages.pop(0)
|
||||||
|
|
||||||
|
|
||||||
|
def comment(cid: int, author: str = "don", body: str = "please fix") -> Comment:
|
||||||
|
return Comment(id=cid, author=author, body=body, created_at="t")
|
||||||
|
|
||||||
|
|
||||||
|
def build(client, **kw) -> tuple[CommentPoller, list[Comment]]:
|
||||||
|
seen: list[Comment] = []
|
||||||
|
poller = CommentPoller(client, "o/r", seen.append, **kw)
|
||||||
|
return poller, seen
|
||||||
|
|
||||||
|
|
||||||
|
def test_new_comments_are_delivered_once() -> None:
|
||||||
|
client = FakeGitea([[comment(1)], [comment(1), comment(2)]])
|
||||||
|
poller, seen = build(client)
|
||||||
|
poller.poll_once(1)
|
||||||
|
poller.poll_once(1)
|
||||||
|
assert [c.id for c in seen] == [1, 2] # id 1 not redelivered
|
||||||
|
|
||||||
|
|
||||||
|
def test_prime_suppresses_the_existing_backlog() -> None:
|
||||||
|
# Starting a watch must not replay feedback already addressed.
|
||||||
|
client = FakeGitea([[comment(1), comment(2)], [comment(1), comment(2), comment(3)]])
|
||||||
|
poller, seen = build(client)
|
||||||
|
poller.prime(1)
|
||||||
|
poller.poll_once(1)
|
||||||
|
assert [c.id for c in seen] == [3]
|
||||||
|
|
||||||
|
|
||||||
|
def test_own_comments_are_ignored() -> None:
|
||||||
|
client = FakeGitea([[comment(1, author="nisse-dev-testing"), comment(2, "don")]])
|
||||||
|
poller, seen = build(client, ignore_authors={"nisse-dev-testing"})
|
||||||
|
poller.poll_once(1)
|
||||||
|
assert [c.id for c in seen] == [2]
|
||||||
|
|
||||||
|
|
||||||
|
def test_ignored_comments_are_not_redelivered_later() -> None:
|
||||||
|
client = FakeGitea(
|
||||||
|
[
|
||||||
|
[comment(1, author="bot")],
|
||||||
|
[comment(1, author="bot"), comment(2, "don")],
|
||||||
|
]
|
||||||
|
)
|
||||||
|
poller, seen = build(client, ignore_authors={"bot"})
|
||||||
|
poller.poll_once(1)
|
||||||
|
poller.poll_once(1)
|
||||||
|
assert [c.id for c in seen] == [2]
|
||||||
|
|
||||||
|
|
||||||
|
def test_outage_does_not_raise_out_of_the_loop() -> None:
|
||||||
|
stream = io.StringIO()
|
||||||
|
client = FakeGitea([[comment(1)]])
|
||||||
|
client.fail_times = 2
|
||||||
|
poller, seen = build(
|
||||||
|
client,
|
||||||
|
oplog=OpLog("nisse-dev", stream),
|
||||||
|
interval_s=0.01,
|
||||||
|
initial_backoff_s=0.01,
|
||||||
|
)
|
||||||
|
poller.start(1)
|
||||||
|
deadline = time.monotonic() + 5
|
||||||
|
while not seen and time.monotonic() < deadline:
|
||||||
|
time.sleep(0.05)
|
||||||
|
poller.stop()
|
||||||
|
|
||||||
|
# It recovered and delivered rather than dying on the outage (B2: an
|
||||||
|
# outage delays a reaction, never fails a task).
|
||||||
|
assert [c.id for c in seen] == [1]
|
||||||
|
assert "gitea.poll_failed" in stream.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
def test_prime_survives_an_outage() -> None:
|
||||||
|
client = FakeGitea()
|
||||||
|
client.fail_times = 1
|
||||||
|
poller, _ = build(client)
|
||||||
|
poller.prime(1) # must not raise
|
||||||
|
|
||||||
|
|
||||||
|
def test_backoff_grows_then_caps() -> None:
|
||||||
|
client = FakeGitea()
|
||||||
|
client.fail_times = 50
|
||||||
|
stream = io.StringIO()
|
||||||
|
poller, _ = build(
|
||||||
|
client,
|
||||||
|
oplog=OpLog("nisse-dev", stream),
|
||||||
|
interval_s=0.001,
|
||||||
|
initial_backoff_s=0.001,
|
||||||
|
max_backoff_s=0.008,
|
||||||
|
)
|
||||||
|
poller.start(1)
|
||||||
|
time.sleep(0.3)
|
||||||
|
poller.stop()
|
||||||
|
|
||||||
|
delays = [json.loads(line)["retry_in_s"] for line in stream.getvalue().splitlines()]
|
||||||
|
assert delays[:4] == [0.001, 0.002, 0.004, 0.008]
|
||||||
|
assert max(delays) == 0.008 # capped, and retried indefinitely (B2)
|
||||||
|
assert MAX_BACKOFF_S <= 300.0 # the shipped cap is "a few minutes"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("author", ["don", "someone-else"])
|
||||||
|
def test_poller_does_not_filter_by_author_itself(author: str) -> None:
|
||||||
|
# Authorisation belongs to the orchestrator's allowlist, not here: the
|
||||||
|
# poller is pure intake.
|
||||||
|
client = FakeGitea([[comment(1, author=author)]])
|
||||||
|
poller, seen = build(client)
|
||||||
|
poller.poll_once(1)
|
||||||
|
assert len(seen) == 1
|
||||||
@@ -0,0 +1,196 @@
|
|||||||
|
"""The B1 loop past EXECUTING: publish, review gate, then next milestone or
|
||||||
|
project completion."""
|
||||||
|
|
||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import AssistantText, CanonicalEvent, StoredEvent, ToolCall
|
||||||
|
from nisse_dev.control import (
|
||||||
|
AWAIT_INPUT,
|
||||||
|
EXECUTING,
|
||||||
|
IDLE,
|
||||||
|
PR_OPEN,
|
||||||
|
Orchestrator,
|
||||||
|
)
|
||||||
|
from nisse_dev.executor import ExecutorOutcome
|
||||||
|
from nisse_dev.gitea import GiteaError, PullRequest
|
||||||
|
from nisse_dev.llm import RoleMap
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.planner import LLMPlanningAgent
|
||||||
|
from nisse_dev.publish import PublishResult
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
|
ROLES = RoleMap(
|
||||||
|
executor="llama3.2",
|
||||||
|
planner="haiku",
|
||||||
|
delegate="haiku",
|
||||||
|
consultant="haiku",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
def __init__(self, plans: list[str]) -> None:
|
||||||
|
self._plans = plans
|
||||||
|
self.calls: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def complete(
|
||||||
|
self,
|
||||||
|
events: list[StoredEvent],
|
||||||
|
tools: list[dict[str, Any]] | None,
|
||||||
|
model: str,
|
||||||
|
system: str | None = None,
|
||||||
|
) -> list[CanonicalEvent]:
|
||||||
|
self.calls.append({"system": system})
|
||||||
|
if not self._plans:
|
||||||
|
return [AssistantText(text="no more plans", model=model)]
|
||||||
|
return [
|
||||||
|
ToolCall(
|
||||||
|
call_id=f"c{len(self.calls)}",
|
||||||
|
tool="submit_plan",
|
||||||
|
arguments={"type": "x", "content": self._plans.pop(0)},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class DoneExecutor:
|
||||||
|
def run(self, milestone: int) -> ExecutorOutcome:
|
||||||
|
return ExecutorOutcome("completed", f"milestone {milestone} done")
|
||||||
|
|
||||||
|
|
||||||
|
class FakePublisher:
|
||||||
|
def __init__(
|
||||||
|
self, *, result: PublishResult | None = None, error: Exception | None = None
|
||||||
|
) -> None:
|
||||||
|
self._result = result or PublishResult(
|
||||||
|
PullRequest(number=7, state="open", head="h", url="https://g/pulls/7"),
|
||||||
|
"opened PR #7: https://g/pulls/7",
|
||||||
|
)
|
||||||
|
self._error = error
|
||||||
|
self.calls: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
def reconcilers(self) -> dict[str, Any]:
|
||||||
|
return {"open_pr": lambda intent: True}
|
||||||
|
|
||||||
|
def publish(self, **kwargs: Any) -> PublishResult:
|
||||||
|
self.calls.append(kwargs)
|
||||||
|
if self._error is not None:
|
||||||
|
raise self._error
|
||||||
|
return self._result
|
||||||
|
|
||||||
|
|
||||||
|
def build(
|
||||||
|
tmp_path: Path, publisher: Any, plans: list[str] | None = None
|
||||||
|
) -> tuple[StateStore, InMemoryTransport]:
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
client = ScriptedClient(plans or ["arch plan", "m1 plan", "m2 plan"])
|
||||||
|
orch = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(client, store, ROLES),
|
||||||
|
operators={"don"},
|
||||||
|
roles=ROLES,
|
||||||
|
executor=DoneExecutor(),
|
||||||
|
publisher=publisher,
|
||||||
|
)
|
||||||
|
orch.start()
|
||||||
|
return store, transport
|
||||||
|
|
||||||
|
|
||||||
|
def run_to_pr(tmp_path: Path, publisher: Any) -> tuple[StateStore, InMemoryTransport]:
|
||||||
|
store, transport = build(tmp_path, publisher)
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve") # arch
|
||||||
|
transport.receive("don", "approve") # milestone plan
|
||||||
|
transport.receive("don", "start milestone 1")
|
||||||
|
return store, transport
|
||||||
|
|
||||||
|
|
||||||
|
def test_completed_milestone_publishes_and_enters_pr_open(tmp_path: Path) -> None:
|
||||||
|
publisher = FakePublisher()
|
||||||
|
store, transport = run_to_pr(tmp_path, publisher)
|
||||||
|
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
|
assert publisher.calls[0]["milestone"] == 1
|
||||||
|
assert publisher.calls[0]["project"] == "demo"
|
||||||
|
assert "m1 plan" in publisher.calls[0]["plan"] # the approved plan travels
|
||||||
|
assert "PR #7" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_pr_open_records_the_pull_request(tmp_path: Path) -> None:
|
||||||
|
store, _ = run_to_pr(tmp_path, FakePublisher())
|
||||||
|
assert store.get_state().data["pr"]["number"] == 7
|
||||||
|
|
||||||
|
|
||||||
|
def test_next_milestone_replans_and_bumps_the_counter(tmp_path: Path) -> None:
|
||||||
|
store, transport = run_to_pr(tmp_path, FakePublisher())
|
||||||
|
transport.receive("don", "next milestone")
|
||||||
|
|
||||||
|
state = store.get_state()
|
||||||
|
assert state.state != PR_OPEN
|
||||||
|
assert state.data["milestone"] == 2
|
||||||
|
assert "m2 plan" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_project_complete_returns_to_idle(tmp_path: Path) -> None:
|
||||||
|
store, transport = run_to_pr(tmp_path, FakePublisher())
|
||||||
|
transport.receive("don", "project complete")
|
||||||
|
|
||||||
|
state = store.get_state()
|
||||||
|
assert state.state == IDLE
|
||||||
|
assert state.project is None
|
||||||
|
assert "project closed" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_command_in_pr_open_explains_the_options(tmp_path: Path) -> None:
|
||||||
|
store, transport = run_to_pr(tmp_path, FakePublisher())
|
||||||
|
transport.receive("don", "looks good!")
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
|
assert "next milestone" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_changes_reports_without_entering_pr_open(tmp_path: Path) -> None:
|
||||||
|
publisher = FakePublisher(result=PublishResult(None, "no changes to publish"))
|
||||||
|
store, transport = run_to_pr(tmp_path, publisher)
|
||||||
|
assert store.get_state().state == EXECUTING
|
||||||
|
assert transport.outbound[-1].text == "no changes to publish"
|
||||||
|
|
||||||
|
|
||||||
|
def test_publish_failure_gates_to_the_operator(tmp_path: Path) -> None:
|
||||||
|
publisher = FakePublisher(error=GiteaError("gitea is down"))
|
||||||
|
store, transport = run_to_pr(tmp_path, publisher)
|
||||||
|
assert store.get_state().state == AWAIT_INPUT
|
||||||
|
assert "could not publish" in transport.outbound[-1].text
|
||||||
|
# And the operator can still get back out (the deadlock regression).
|
||||||
|
transport.receive("don", "gitea is back")
|
||||||
|
assert store.get_state().state != AWAIT_INPUT
|
||||||
|
|
||||||
|
|
||||||
|
def test_publisher_reconciler_is_registered_at_boot(tmp_path: Path) -> None:
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
store.begin_intent("open_pr", {"branch": "b"}, cause="crash")
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
orch = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(ScriptedClient([]), store, ROLES),
|
||||||
|
operators={"don"},
|
||||||
|
roles=ROLES,
|
||||||
|
publisher=FakePublisher(),
|
||||||
|
)
|
||||||
|
orch.start()
|
||||||
|
# The publisher's reconciler answered "already opened", so boot resumed
|
||||||
|
# instead of gating on an unknown intent kind.
|
||||||
|
assert store.pending_intents() == []
|
||||||
|
assert store.get_state().state != AWAIT_INPUT
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("command", ["next milestone", "project complete"])
|
||||||
|
def test_pr_open_commands_are_operator_only(tmp_path: Path, command: str) -> None:
|
||||||
|
store, transport = run_to_pr(tmp_path, FakePublisher())
|
||||||
|
transport.receive("mallory", command)
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.git import GitRepo
|
||||||
|
from nisse_dev.gitea import GiteaError, PullRequest
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.publish import OPEN_PR_INTENT, MilestonePublisher
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
|
||||||
|
|
||||||
|
class FakeGitea:
|
||||||
|
def __init__(self, *, fail: bool = False) -> None:
|
||||||
|
self.opened: list[dict[str, Any]] = []
|
||||||
|
self.existing: list[str] = []
|
||||||
|
self._fail = fail
|
||||||
|
|
||||||
|
def open_pull_request(
|
||||||
|
self, repo: str, *, head: str, base: str, title: str, body: str
|
||||||
|
) -> PullRequest:
|
||||||
|
if self._fail:
|
||||||
|
raise GiteaError("gitea is down")
|
||||||
|
self.opened.append(
|
||||||
|
{"repo": repo, "head": head, "base": base, "title": title, "body": body}
|
||||||
|
)
|
||||||
|
self.existing.append(head)
|
||||||
|
return PullRequest(
|
||||||
|
number=len(self.opened), state="open", head=head, url=f"u/{head}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def find_pull_request(self, repo: str, head: str) -> PullRequest | None:
|
||||||
|
if head in self.existing:
|
||||||
|
return PullRequest(number=1, state="open", head=head, url="u")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class FakeGit:
|
||||||
|
"""Records what would have been pushed; no network."""
|
||||||
|
|
||||||
|
def __init__(self, root: Path, *, changes: bool = True) -> None:
|
||||||
|
self.repo = GitRepo(root)
|
||||||
|
self.repo.init()
|
||||||
|
self.repo.configure_identity("bot", "bot@example.test")
|
||||||
|
if changes:
|
||||||
|
(root / "work.txt").write_text("done")
|
||||||
|
self.pushed: list[str] = []
|
||||||
|
|
||||||
|
def current_branch(self) -> str:
|
||||||
|
return self.repo.current_branch()
|
||||||
|
|
||||||
|
def checkout_new_branch(self, name: str) -> None:
|
||||||
|
self.repo.checkout_new_branch(name)
|
||||||
|
|
||||||
|
def commit_all(self, message: str) -> str | None:
|
||||||
|
return self.repo.commit_all(message)
|
||||||
|
|
||||||
|
def push(self, remote_url: str, branch: str, *, token: str | None = None) -> None:
|
||||||
|
self.pushed.append(branch)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path: Path) -> StateStore:
|
||||||
|
return StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
|
||||||
|
|
||||||
|
def build(store: StateStore, git: Any, client: Any) -> MilestonePublisher:
|
||||||
|
return MilestonePublisher(
|
||||||
|
store,
|
||||||
|
client,
|
||||||
|
"o/sandbox",
|
||||||
|
git,
|
||||||
|
base_url="https://gitea.test",
|
||||||
|
account="bot",
|
||||||
|
token="tok",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_publish_commits_pushes_and_opens_a_pr(
|
||||||
|
store: StateStore, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
git = FakeGit(tmp_path / "work")
|
||||||
|
client = FakeGitea()
|
||||||
|
result = build(store, git, client).publish(
|
||||||
|
task_id="demo-abc", milestone=2, project="demo", plan="the plan"
|
||||||
|
)
|
||||||
|
|
||||||
|
assert git.pushed == ["demo-abc/milestone-2"]
|
||||||
|
assert client.opened[0]["head"] == "demo-abc/milestone-2"
|
||||||
|
assert client.opened[0]["base"] == "main"
|
||||||
|
assert "the plan" in client.opened[0]["body"]
|
||||||
|
assert result.pull_request is not None
|
||||||
|
assert "opened PR #1" in result.note
|
||||||
|
|
||||||
|
|
||||||
|
def test_intent_is_recorded_then_committed(store: StateStore, tmp_path: Path) -> None:
|
||||||
|
build(store, FakeGit(tmp_path / "work"), FakeGitea()).publish(
|
||||||
|
task_id="t", milestone=1, project="demo", plan="p"
|
||||||
|
)
|
||||||
|
# Nothing left pending: the intent opened before the push was settled
|
||||||
|
# once the PR existed.
|
||||||
|
assert store.pending_intents() == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_failure_after_intent_leaves_it_pending_for_reconciliation(
|
||||||
|
store: StateStore, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
publisher = build(store, FakeGit(tmp_path / "work"), FakeGitea(fail=True))
|
||||||
|
with pytest.raises(GiteaError):
|
||||||
|
publisher.publish(task_id="t", milestone=1, project="demo", plan="p")
|
||||||
|
|
||||||
|
[pending] = store.pending_intents()
|
||||||
|
assert pending.kind == OPEN_PR_INTENT
|
||||||
|
assert pending.payload["branch"] == "t/milestone-1"
|
||||||
|
assert pending.payload["sha"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_nothing_to_commit_is_reported_not_raised(
|
||||||
|
store: StateStore, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
git = FakeGit(tmp_path / "work", changes=False)
|
||||||
|
client = FakeGitea()
|
||||||
|
result = build(store, git, client).publish(
|
||||||
|
task_id="t", milestone=1, project="demo", plan="p"
|
||||||
|
)
|
||||||
|
assert result.pull_request is None
|
||||||
|
assert result.note == "no changes to publish"
|
||||||
|
assert client.opened == []
|
||||||
|
# No externally visible act was attempted, so no intent was opened.
|
||||||
|
assert store.pending_intents() == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_branch_is_namespaced_by_task(store: StateStore, tmp_path: Path) -> None:
|
||||||
|
publisher = build(store, FakeGit(tmp_path / "work"), FakeGitea())
|
||||||
|
assert publisher.branch_for("greet-9f2", 3) == "greet-9f2/milestone-3"
|
||||||
|
|
||||||
|
|
||||||
|
def test_reconciler_settles_an_intent_left_by_a_crash(
|
||||||
|
store: StateStore, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
# Simulate the dangerous window: the PR was opened, but the process died
|
||||||
|
# before the intent could be committed. On the next boot the reconciler
|
||||||
|
# must say "already done" so the milestone is not published twice.
|
||||||
|
client = FakeGitea()
|
||||||
|
publisher = build(store, FakeGit(tmp_path / "work"), client)
|
||||||
|
intent_id = store.begin_intent(
|
||||||
|
OPEN_PR_INTENT, {"branch": "t/milestone-1"}, cause="crash simulation"
|
||||||
|
)
|
||||||
|
client.existing.append("t/milestone-1")
|
||||||
|
|
||||||
|
reconcilers = publisher.reconcilers()
|
||||||
|
assert set(reconcilers) == {OPEN_PR_INTENT}
|
||||||
|
[pending] = store.pending_intents()
|
||||||
|
assert reconcilers[OPEN_PR_INTENT](pending) is True
|
||||||
|
|
||||||
|
store.commit_intent(intent_id, cause="verified")
|
||||||
|
assert client.opened == [] # nothing re-published
|
||||||
@@ -71,6 +71,33 @@ def test_compaction_replaces_its_range_only() -> None:
|
|||||||
assert contents == ["kept", "[summary of earlier conversation]\nthe old stuff"]
|
assert contents == ["kept", "[summary of earlier conversation]\nthe old stuff"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_call_without_a_result_gets_a_placeholder() -> None:
|
||||||
|
# Providers reject a dangling tool_use; this happens after a crash (A3)
|
||||||
|
# or with turn-ending gate tools like submit_plan.
|
||||||
|
events = stored(
|
||||||
|
[
|
||||||
|
UserMessage(text="go", source="m:1"),
|
||||||
|
ToolCall(call_id="call-x", tool="submit_plan", arguments={}),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
body = OpenAIChatRenderer().render(events, tools=None, model="q")
|
||||||
|
assert [m["role"] for m in body["messages"]] == ["user", "assistant", "tool"]
|
||||||
|
assert body["messages"][2]["tool_call_id"] == "call-x"
|
||||||
|
assert body["messages"][2]["content"] == "[no result recorded]"
|
||||||
|
|
||||||
|
|
||||||
|
def test_answered_tool_call_gets_no_placeholder() -> None:
|
||||||
|
events = stored(
|
||||||
|
[
|
||||||
|
ToolCall(call_id="call-x", tool="run_command", arguments={}),
|
||||||
|
ToolResult(call_id="call-x", result="ok", success=True),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
body = OpenAIChatRenderer().render(events, tools=None, model="q")
|
||||||
|
assert [m["role"] for m in body["messages"]] == ["assistant", "tool"]
|
||||||
|
assert body["messages"][1]["content"] == "ok"
|
||||||
|
|
||||||
|
|
||||||
def test_gate_event_renders_as_context() -> None:
|
def test_gate_event_renders_as_context() -> None:
|
||||||
events = stored([GateOpened(gate="arch", reason="plan submitted (rev-1)")])
|
events = stored([GateOpened(gate="arch", reason="plan submitted (rev-1)")])
|
||||||
body = OpenAIChatRenderer().render(events, tools=None, model="q")
|
body = OpenAIChatRenderer().render(events, tools=None, model="q")
|
||||||
|
|||||||
@@ -0,0 +1,196 @@
|
|||||||
|
"""The Anthropic Messages shape differs structurally from the OpenAI one;
|
||||||
|
these tests pin the differences that a normalising middlebox would blur."""
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import (
|
||||||
|
AssistantText,
|
||||||
|
Compaction,
|
||||||
|
StoredEvent,
|
||||||
|
ToolCall,
|
||||||
|
ToolResult,
|
||||||
|
UserMessage,
|
||||||
|
)
|
||||||
|
from nisse_dev.llm import AnthropicPassthroughClient, LLMError
|
||||||
|
from nisse_dev.render_anthropic import (
|
||||||
|
AnthropicMessagesRenderer,
|
||||||
|
to_anthropic_tools,
|
||||||
|
)
|
||||||
|
|
||||||
|
TOOLS = [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "run_command",
|
||||||
|
"description": "Run a shell command.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"command": {"type": "string"}},
|
||||||
|
"required": ["command"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def stored(events: list) -> list[StoredEvent]:
|
||||||
|
return [StoredEvent(id=i + 1, ts="t", event=e) for i, e in enumerate(events)]
|
||||||
|
|
||||||
|
|
||||||
|
def render(events: list, **kw) -> dict:
|
||||||
|
return AnthropicMessagesRenderer().render(
|
||||||
|
stored(events), kw.pop("tools", None), "claude-haiku-4-5", **kw
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_system_is_a_top_level_field_not_a_message() -> None:
|
||||||
|
body = render([UserMessage(text="hi", source="m")], system="you are nisse")
|
||||||
|
assert body["system"] == "you are nisse"
|
||||||
|
assert all(m["role"] != "system" for m in body["messages"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_max_tokens_is_always_present() -> None:
|
||||||
|
# The Messages API rejects a request without it.
|
||||||
|
assert render([UserMessage(text="hi", source="m")])["max_tokens"] > 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_call_is_a_content_block_not_an_array() -> None:
|
||||||
|
body = render(
|
||||||
|
[ToolCall(call_id="call-1", tool="run_command", arguments={"command": "ls"})]
|
||||||
|
)
|
||||||
|
block = body["messages"][0]["content"][0]
|
||||||
|
assert block["type"] == "tool_use"
|
||||||
|
assert block["id"] == "call-1"
|
||||||
|
assert block["input"] == {"command": "ls"}
|
||||||
|
assert "tool_calls" not in body["messages"][0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_tool_result_is_a_user_block_directly_after_the_call() -> None:
|
||||||
|
body = render(
|
||||||
|
[
|
||||||
|
ToolCall(call_id="call-1", tool="run_command", arguments={}),
|
||||||
|
ToolResult(call_id="call-1", result="exit code: 0", success=True),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assistant, user = body["messages"]
|
||||||
|
assert assistant["role"] == "assistant"
|
||||||
|
assert assistant["content"][0]["type"] == "tool_use"
|
||||||
|
assert user["role"] == "user" # not a "tool" role, as OpenAI would have
|
||||||
|
assert user["content"][0] == {
|
||||||
|
"type": "tool_result",
|
||||||
|
"tool_use_id": "call-1",
|
||||||
|
"content": "exit code: 0",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_adjacent_same_role_events_are_merged() -> None:
|
||||||
|
body = render(
|
||||||
|
[
|
||||||
|
UserMessage(text="one", source="m"),
|
||||||
|
UserMessage(text="two", source="m"),
|
||||||
|
AssistantText(text="reply", model="m"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert [m["role"] for m in body["messages"]] == ["user", "assistant"]
|
||||||
|
assert [b["text"] for b in body["messages"][0]["content"]] == ["one", "two"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_dangling_tool_call_gets_a_placeholder_result() -> None:
|
||||||
|
body = render([ToolCall(call_id="call-x", tool="submit_plan", arguments={})])
|
||||||
|
assert [m["role"] for m in body["messages"]] == ["assistant", "user"]
|
||||||
|
assert body["messages"][1]["content"][0]["tool_use_id"] == "call-x"
|
||||||
|
|
||||||
|
|
||||||
|
def test_tools_are_converted_to_input_schema() -> None:
|
||||||
|
[tool] = to_anthropic_tools(TOOLS)
|
||||||
|
assert tool["name"] == "run_command"
|
||||||
|
assert tool["input_schema"]["properties"] == {"command": {"type": "string"}}
|
||||||
|
assert "parameters" not in tool
|
||||||
|
assert "function" not in tool
|
||||||
|
|
||||||
|
|
||||||
|
def test_compaction_still_replaces_its_range() -> None:
|
||||||
|
events = stored(
|
||||||
|
[
|
||||||
|
UserMessage(text="old", source="m"),
|
||||||
|
UserMessage(text="kept", source="m"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
events.append(
|
||||||
|
StoredEvent(
|
||||||
|
id=3,
|
||||||
|
ts="t",
|
||||||
|
event=Compaction(summary="summary", model="m", replaces=[1, 1]),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
body = AnthropicMessagesRenderer().render(events, None, "claude-haiku-4-5")
|
||||||
|
texts = [b["text"] for m in body["messages"] for b in m["content"]]
|
||||||
|
assert "old" not in texts
|
||||||
|
assert any("summary" in t for t in texts)
|
||||||
|
|
||||||
|
|
||||||
|
# --- passthrough client --------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def client(handler) -> AnthropicPassthroughClient:
|
||||||
|
return AnthropicPassthroughClient(
|
||||||
|
"https://litellm.test", "tok", transport=httpx.MockTransport(handler)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_client_posts_to_the_messages_route_with_version_header() -> None:
|
||||||
|
seen: dict = {}
|
||||||
|
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
seen["url"] = str(request.url)
|
||||||
|
seen["version"] = request.headers.get("anthropic-version")
|
||||||
|
return httpx.Response(
|
||||||
|
200,
|
||||||
|
json={
|
||||||
|
"model": "claude-haiku-4-5",
|
||||||
|
"content": [{"type": "text", "text": "hi"}],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
events = stored([UserMessage(text="hello", source="m")])
|
||||||
|
[event] = client(handler).complete(events, None, "claude-haiku-4-5")
|
||||||
|
assert seen["url"] == "https://litellm.test/v1/messages"
|
||||||
|
assert seen["version"] == "2023-06-01"
|
||||||
|
assert event == AssistantText(text="hi", model="claude-haiku-4-5")
|
||||||
|
|
||||||
|
|
||||||
|
def test_client_maps_tool_use_to_local_call_ids() -> None:
|
||||||
|
response = {
|
||||||
|
"model": "claude-haiku-4-5",
|
||||||
|
"content": [
|
||||||
|
{"type": "text", "text": "running it"},
|
||||||
|
{
|
||||||
|
"type": "tool_use",
|
||||||
|
"id": "toolu_vendor_xyz",
|
||||||
|
"name": "run_command",
|
||||||
|
"input": {"command": "pytest"},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
events = stored([UserMessage(text="go", source="m")])
|
||||||
|
text, call = client(lambda r: httpx.Response(200, json=response)).complete(
|
||||||
|
events, TOOLS, "claude-haiku-4-5"
|
||||||
|
)
|
||||||
|
assert isinstance(text, AssistantText)
|
||||||
|
assert isinstance(call, ToolCall)
|
||||||
|
assert call.arguments == {"command": "pytest"}
|
||||||
|
assert call.call_id != "toolu_vendor_xyz" # provider id never a join key
|
||||||
|
assert call.call_id.startswith("call-")
|
||||||
|
|
||||||
|
|
||||||
|
def test_client_errors_are_llm_errors() -> None:
|
||||||
|
events = stored([UserMessage(text="go", source="m")])
|
||||||
|
with pytest.raises(LLMError, match="400"):
|
||||||
|
client(lambda r: httpx.Response(400, text="bad")).complete(
|
||||||
|
events, None, "claude-haiku-4-5"
|
||||||
|
)
|
||||||
|
with pytest.raises(LLMError, match="malformed"):
|
||||||
|
client(lambda r: httpx.Response(200, json={"oops": 1})).complete(
|
||||||
|
events, None, "claude-haiku-4-5"
|
||||||
|
)
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""Reacting to PR review comments while in PR_OPEN (spec B2)."""
|
||||||
|
|
||||||
|
import io
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from nisse_dev.canonical import CanonicalEvent, StoredEvent, ToolCall
|
||||||
|
from nisse_dev.control import AWAIT_INPUT, PR_OPEN, Orchestrator
|
||||||
|
from nisse_dev.executor import ExecutorOutcome
|
||||||
|
from nisse_dev.gitea import Comment, GiteaError, PullRequest
|
||||||
|
from nisse_dev.llm import RoleMap
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.planner import LLMPlanningAgent
|
||||||
|
from nisse_dev.publish import PublishResult
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
|
ROLES = RoleMap(executor="x", planner="y", delegate="y", consultant="y")
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedClient:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.calls = 0
|
||||||
|
|
||||||
|
def complete(self, events, tools, model, system=None) -> list[CanonicalEvent]:
|
||||||
|
self.calls += 1
|
||||||
|
return [
|
||||||
|
ToolCall(
|
||||||
|
call_id=f"c{self.calls}",
|
||||||
|
tool="submit_plan",
|
||||||
|
arguments={"type": "x", "content": f"plan {self.calls}"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingExecutor:
|
||||||
|
def __init__(self, outcomes: list[str] | None = None) -> None:
|
||||||
|
self.runs: list[int] = []
|
||||||
|
self.seen_events: list[list[StoredEvent]] = []
|
||||||
|
# One entry per run, so a test can let the milestone succeed and
|
||||||
|
# only the review response fail.
|
||||||
|
self._outcomes = outcomes or []
|
||||||
|
self.store: StateStore | None = None
|
||||||
|
|
||||||
|
def run(self, milestone: int) -> ExecutorOutcome:
|
||||||
|
self.runs.append(milestone)
|
||||||
|
if self.store is not None:
|
||||||
|
self.seen_events.append(self.store.read_events())
|
||||||
|
status = self._outcomes.pop(0) if self._outcomes else "completed"
|
||||||
|
return ExecutorOutcome(status, f"executor said {status}")
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingPublisher:
|
||||||
|
def __init__(self, *, sha: str | None = "abc12345", error: Exception | None = None):
|
||||||
|
self._sha = sha
|
||||||
|
self._error = error
|
||||||
|
self.followups: list[dict[str, Any]] = []
|
||||||
|
self.comments: list[tuple[int, str]] = []
|
||||||
|
|
||||||
|
def reconcilers(self) -> dict[str, Any]:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
def publish(self, **kwargs: Any) -> PublishResult:
|
||||||
|
return PublishResult(
|
||||||
|
PullRequest(number=42, state="open", head="h", url="u"), "opened PR #42: u"
|
||||||
|
)
|
||||||
|
|
||||||
|
def push_followup(self, **kwargs: Any) -> str | None:
|
||||||
|
self.followups.append(kwargs)
|
||||||
|
if self._error is not None:
|
||||||
|
raise self._error
|
||||||
|
return self._sha
|
||||||
|
|
||||||
|
def comment(self, index: int, body: str) -> None:
|
||||||
|
self.comments.append((index, body))
|
||||||
|
|
||||||
|
|
||||||
|
def build(
|
||||||
|
tmp_path: Path,
|
||||||
|
executor: Any,
|
||||||
|
publisher: Any,
|
||||||
|
gitea_operators: set[str] | None = None,
|
||||||
|
) -> tuple[Orchestrator, StateStore, InMemoryTransport]:
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
if hasattr(executor, "store"):
|
||||||
|
executor.store = store
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
orch = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(ScriptedClient(), store, ROLES),
|
||||||
|
operators={"don"},
|
||||||
|
roles=ROLES,
|
||||||
|
executor=executor,
|
||||||
|
publisher=publisher,
|
||||||
|
gitea_operators=gitea_operators if gitea_operators is not None else {"don"},
|
||||||
|
)
|
||||||
|
orch.start()
|
||||||
|
transport.receive("don", "use demo")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
transport.receive("don", "start milestone 1")
|
||||||
|
return orch, store, transport
|
||||||
|
|
||||||
|
|
||||||
|
def review(
|
||||||
|
cid: int = 5, author: str = "don", body: str = "rename the module"
|
||||||
|
) -> Comment:
|
||||||
|
return Comment(id=cid, author=author, body=body, created_at="t")
|
||||||
|
|
||||||
|
|
||||||
|
def test_review_comment_triggers_a_fix_and_a_push(tmp_path: Path) -> None:
|
||||||
|
executor, publisher = RecordingExecutor(), RecordingPublisher()
|
||||||
|
orch, store, transport = build(tmp_path, executor, publisher)
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
|
|
||||||
|
orch.handle_review_comment(review())
|
||||||
|
|
||||||
|
assert executor.runs == [1, 1] # milestone run, then the review response
|
||||||
|
assert publisher.followups[0]["milestone"] == 1
|
||||||
|
assert "abc12345" in transport.outbound[-1].text
|
||||||
|
assert publisher.comments[0][0] == 42 # replied on the PR
|
||||||
|
|
||||||
|
|
||||||
|
def test_comment_enters_the_transcript_before_the_executor_runs(
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
executor, publisher = RecordingExecutor(), RecordingPublisher()
|
||||||
|
orch, _, _ = build(tmp_path, executor, publisher)
|
||||||
|
orch.handle_review_comment(review(body="please add a test"))
|
||||||
|
|
||||||
|
# The executor must see the feedback as conversation, not as an
|
||||||
|
# out-of-band instruction it was never shown.
|
||||||
|
texts = [getattr(e.event, "text", "") for e in executor.seen_events[-1]]
|
||||||
|
assert any("please add a test" in t for t in texts)
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_operator_comments_are_ignored(tmp_path: Path) -> None:
|
||||||
|
executor, publisher = RecordingExecutor(), RecordingPublisher()
|
||||||
|
orch, store, _ = build(tmp_path, executor, publisher)
|
||||||
|
orch.handle_review_comment(review(author="drive-by"))
|
||||||
|
|
||||||
|
assert executor.runs == [1] # only the original milestone run
|
||||||
|
assert publisher.followups == []
|
||||||
|
assert store.get_state().state == PR_OPEN
|
||||||
|
|
||||||
|
|
||||||
|
def test_comments_outside_pr_open_are_ignored(tmp_path: Path) -> None:
|
||||||
|
executor, publisher = RecordingExecutor(), RecordingPublisher()
|
||||||
|
orch, _store, transport = build(tmp_path, executor, publisher)
|
||||||
|
transport.receive("don", "project complete") # back to IDLE
|
||||||
|
|
||||||
|
orch.handle_review_comment(review())
|
||||||
|
assert executor.runs == [1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_failed_review_response_gates_to_the_operator(tmp_path: Path) -> None:
|
||||||
|
# Milestone succeeds; the response to the review comment does not.
|
||||||
|
executor = RecordingExecutor(outcomes=["completed", "escalate"])
|
||||||
|
publisher = RecordingPublisher()
|
||||||
|
orch, store, transport = build(tmp_path, executor, publisher)
|
||||||
|
orch.handle_review_comment(review())
|
||||||
|
|
||||||
|
assert store.get_state().state == AWAIT_INPUT
|
||||||
|
assert "could not address the review comment" in transport.outbound[-1].text
|
||||||
|
assert publisher.followups == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_code_change_needed_is_reported(tmp_path: Path) -> None:
|
||||||
|
executor, publisher = RecordingExecutor(), RecordingPublisher(sha=None)
|
||||||
|
orch, _, transport = build(tmp_path, executor, publisher)
|
||||||
|
orch.handle_review_comment(review(body="just curious, why sqlite?"))
|
||||||
|
|
||||||
|
assert "no code change was needed" in transport.outbound[-1].text
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_failure_is_reported_without_losing_pr_open(tmp_path: Path) -> None:
|
||||||
|
executor = RecordingExecutor()
|
||||||
|
publisher = RecordingPublisher(error=GiteaError("push rejected"))
|
||||||
|
orch, store, transport = build(tmp_path, executor, publisher)
|
||||||
|
orch.handle_review_comment(review())
|
||||||
|
|
||||||
|
assert "could not push the fix" in transport.outbound[-1].text
|
||||||
|
assert store.get_state().state == PR_OPEN # still reviewable, can retry
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
import os
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.tools import EXECUTOR_TOOLS, MAX_OUTPUT_CHARS, Workspace
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def ws(tmp_path: Path) -> Workspace:
|
||||||
|
return Workspace(tmp_path / "work", timeout_s=10)
|
||||||
|
|
||||||
|
|
||||||
|
def test_manifest_is_small_and_flat() -> None:
|
||||||
|
# A15: fewer tools, flatter arguments for the local executor.
|
||||||
|
names = [t["function"]["name"] for t in EXECUTOR_TOOLS]
|
||||||
|
assert names == ["read_file", "write_file", "list_dir", "run_command"]
|
||||||
|
for tool in EXECUTOR_TOOLS:
|
||||||
|
props = tool["function"]["parameters"]["properties"]
|
||||||
|
assert all(p["type"] == "string" for p in props.values())
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_then_read_round_trip(ws: Workspace) -> None:
|
||||||
|
assert ws.write_file("src/app.py", "print('hi')\n").success
|
||||||
|
outcome = ws.read_file("src/app.py")
|
||||||
|
assert outcome.success
|
||||||
|
assert outcome.result == "print('hi')\n"
|
||||||
|
|
||||||
|
|
||||||
|
def test_relative_paths_land_in_the_workspace(ws: Workspace) -> None:
|
||||||
|
ws.write_file("notes.txt", "x")
|
||||||
|
assert (ws.root / "notes.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_missing_file_is_a_failed_result_not_an_exception(
|
||||||
|
ws: Workspace,
|
||||||
|
) -> None:
|
||||||
|
outcome = ws.read_file("nope.txt")
|
||||||
|
assert not outcome.success
|
||||||
|
assert "could not read" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_dir(ws: Workspace) -> None:
|
||||||
|
ws.write_file("a.txt", "a")
|
||||||
|
ws.write_file("sub/b.txt", "b")
|
||||||
|
outcome = ws.list_dir(".")
|
||||||
|
assert outcome.success
|
||||||
|
assert outcome.result.splitlines() == ["a.txt", "sub/"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_empty_dir(ws: Workspace) -> None:
|
||||||
|
assert ws.list_dir(".").result == "(empty)"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_command_success_reports_exit_code_and_stdout(ws: Workspace) -> None:
|
||||||
|
outcome = ws.run_command("echo hello")
|
||||||
|
assert outcome.success
|
||||||
|
assert "exit code: 0" in outcome.result
|
||||||
|
assert "hello" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_nonzero_exit_is_a_result_not_an_error(ws: Workspace) -> None:
|
||||||
|
outcome = ws.run_command("exit 3")
|
||||||
|
assert not outcome.success # feeds the A12 failure count
|
||||||
|
assert "exit code: 3" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_command_runs_in_the_workspace(ws: Workspace) -> None:
|
||||||
|
ws.write_file("marker.txt", "here")
|
||||||
|
assert "marker.txt" in ws.run_command("ls").result
|
||||||
|
|
||||||
|
|
||||||
|
def test_command_timeout_is_reported(tmp_path: Path) -> None:
|
||||||
|
ws = Workspace(tmp_path / "work", timeout_s=1)
|
||||||
|
outcome = ws.run_command("sleep 30")
|
||||||
|
assert not outcome.success
|
||||||
|
assert "timed out" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def _is_running(pid: int) -> bool:
|
||||||
|
"""os.kill(pid, 0) is not enough: a killed orphan stays a zombie until
|
||||||
|
something reaps it, and no init does that inside a CI container. Read the
|
||||||
|
process state instead."""
|
||||||
|
try:
|
||||||
|
stat = Path(f"/proc/{pid}/stat").read_text()
|
||||||
|
except FileNotFoundError:
|
||||||
|
return False
|
||||||
|
# State is the first field after the parenthesised comm.
|
||||||
|
return stat.rsplit(")", 1)[1].split()[0] != "Z"
|
||||||
|
|
||||||
|
|
||||||
|
def test_timeout_kills_the_whole_process_group(tmp_path: Path) -> None:
|
||||||
|
ws = Workspace(tmp_path / "work", timeout_s=1)
|
||||||
|
pidfile = ws.root / "child.pid"
|
||||||
|
# A backgrounded grandchild must not outlive the timeout.
|
||||||
|
ws.run_command(f"(sleep 30 & echo $! > {pidfile}); sleep 30")
|
||||||
|
assert pidfile.exists()
|
||||||
|
pid = int(pidfile.read_text().strip())
|
||||||
|
assert _is_running(os.getpid()), "liveness check is broken, not the kill"
|
||||||
|
# SIGKILL to a process group is asynchronous, and the grandchild is an
|
||||||
|
# orphan by now, so its exit is not synchronised with our return. Wait
|
||||||
|
# for death rather than assuming it is instantaneous.
|
||||||
|
deadline = time.monotonic() + 5
|
||||||
|
while _is_running(pid) and time.monotonic() < deadline:
|
||||||
|
time.sleep(0.05)
|
||||||
|
assert not _is_running(pid)
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_credentials_are_not_inherited(
|
||||||
|
ws: Workspace, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setenv("NISSE_LLM_TOKEN", "sk-secret")
|
||||||
|
monkeypatch.setenv("PATH", os.environ["PATH"])
|
||||||
|
outcome = ws.run_command("echo [${NISSE_LLM_TOKEN}]")
|
||||||
|
assert "sk-secret" not in outcome.result
|
||||||
|
assert "[]" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_stdin_is_closed_so_prompts_fail_fast(ws: Workspace) -> None:
|
||||||
|
outcome = ws.run_command("read line && echo got")
|
||||||
|
assert not outcome.success
|
||||||
|
assert "got" not in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_long_output_is_truncated(ws: Workspace) -> None:
|
||||||
|
outcome = ws.run_command("printf 'x%.0s' $(seq 1 20000)")
|
||||||
|
assert len(outcome.result) < MAX_OUTPUT_CHARS + 200
|
||||||
|
assert "truncated" in outcome.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_dispatch_unknown_tool_and_missing_argument(ws: Workspace) -> None:
|
||||||
|
assert not ws.dispatch("fly", {}).success
|
||||||
|
missing = ws.dispatch("read_file", {})
|
||||||
|
assert not missing.success
|
||||||
|
assert "missing argument" in missing.result
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_outside_the_workspace_is_refused(ws: Workspace, tmp_path: Path) -> None:
|
||||||
|
# Not sandboxing: a file written outside the repo is invisible to git, so
|
||||||
|
# the work would vanish while the model truthfully reported success.
|
||||||
|
# Models really do invent absolute paths like /workspace.
|
||||||
|
stray = tmp_path / "elsewhere" / "slugify.py"
|
||||||
|
outcome = ws.write_file(str(stray), "x")
|
||||||
|
assert not outcome.success
|
||||||
|
assert "outside the workspace" in outcome.result
|
||||||
|
assert str(ws.root) in outcome.result # tells it where to write instead
|
||||||
|
assert not stray.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_escaping_via_parent_traversal_is_refused(ws: Workspace) -> None:
|
||||||
|
outcome = ws.write_file("../escaped.txt", "x")
|
||||||
|
assert not outcome.success
|
||||||
|
assert not (ws.root.parent / "escaped.txt").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_describe_names_the_root(ws: Workspace) -> None:
|
||||||
|
assert str(ws.root) in ws.describe()
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
"""A real delegation over the Anthropic passthrough: scoped sub-task in,
|
||||||
|
working code and a structured handback out."""
|
||||||
|
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.delegation import Caps, Delegation
|
||||||
|
from nisse_dev.envfile import load_env_file
|
||||||
|
from nisse_dev.git import GitRepo
|
||||||
|
from nisse_dev.llm import AnthropicPassthroughClient
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import Workspace
|
||||||
|
|
||||||
|
load_env_file(Path(__file__).resolve().parents[2] / ".env")
|
||||||
|
|
||||||
|
SENTINEL = "NISSE_LIVE_PLANNER_MODEL"
|
||||||
|
|
||||||
|
pytestmark = [
|
||||||
|
pytest.mark.integration,
|
||||||
|
pytest.mark.skipif(
|
||||||
|
not os.environ.get(SENTINEL),
|
||||||
|
reason=f"no live frontier route configured ({SENTINEL} unset)",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
BRIEF = """Create `slugify.py` in the workspace root containing a function
|
||||||
|
`slugify(text)` that lowercases the text and replaces every run of
|
||||||
|
non-alphanumeric characters with a single hyphen, with no leading or trailing
|
||||||
|
hyphen. Then create `test_slugify.py` with at least two assertions covering
|
||||||
|
it, and run `python -m pytest test_slugify.py -q` until it passes."""
|
||||||
|
|
||||||
|
|
||||||
|
def build(tmp_path: Path, caps: Caps) -> tuple[Delegation, GitRepo, StateStore]:
|
||||||
|
repo = GitRepo(tmp_path / "work")
|
||||||
|
repo.init()
|
||||||
|
repo.configure_identity("nisse-dev-testing", "n@t.test")
|
||||||
|
(repo.root / "README.md").write_text("sandbox\n")
|
||||||
|
repo.commit_all("seed")
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
delegation = Delegation(
|
||||||
|
AnthropicPassthroughClient.from_env(),
|
||||||
|
store,
|
||||||
|
Workspace(repo.root, timeout_s=60),
|
||||||
|
repo,
|
||||||
|
model=os.environ[SENTINEL],
|
||||||
|
caps=caps,
|
||||||
|
)
|
||||||
|
return delegation, repo, store
|
||||||
|
|
||||||
|
|
||||||
|
def test_delegate_completes_a_scoped_subtask(tmp_path: Path) -> None:
|
||||||
|
delegation, repo, _store = build(tmp_path, Caps(turns=12, wall_clock_s=300.0))
|
||||||
|
|
||||||
|
result = delegation.run(brief=BRIEF, branch="live/delegation-1")
|
||||||
|
|
||||||
|
assert result.usable, f"delegation did not hand back: {result.status} {result.note}"
|
||||||
|
assert (repo.root / "slugify.py").exists()
|
||||||
|
|
||||||
|
# The handback is the entire interface back, so it must actually say
|
||||||
|
# something in each part rather than be structurally present but empty.
|
||||||
|
handback = result.handback
|
||||||
|
assert handback is not None
|
||||||
|
assert len(handback.changed) > 10
|
||||||
|
assert len(handback.now_true) > 10
|
||||||
|
assert len(handback.next) > 5
|
||||||
|
|
||||||
|
# The delegate's own work is on its branch, and it really runs.
|
||||||
|
assert repo.current_branch() == "live/delegation-1"
|
||||||
|
check = Workspace(repo.root, timeout_s=60).run_command(
|
||||||
|
"python -m pytest test_slugify.py -q"
|
||||||
|
)
|
||||||
|
assert check.success, f"delegate's own tests do not pass: {check.result[:400]}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_cap_breach_discards_the_work(tmp_path: Path) -> None:
|
||||||
|
# One turn cannot finish the brief, so this exercises the abort path
|
||||||
|
# against a real model rather than a scripted one.
|
||||||
|
delegation, repo, _ = build(tmp_path, Caps(turns=1, wall_clock_s=120.0))
|
||||||
|
|
||||||
|
result = delegation.run(brief=BRIEF, branch="live/delegation-2")
|
||||||
|
|
||||||
|
assert not result.usable
|
||||||
|
assert result.status == "capped"
|
||||||
|
assert repo.current_branch() == "main"
|
||||||
|
assert not (repo.root / "slugify.py").exists()
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
"""Live Gitea round trip against the sandbox repo: push a branch, open a PR,
|
||||||
|
comment, and prove the A3 reconciler reads external truth correctly.
|
||||||
|
|
||||||
|
Cleans up after itself (closes the PR, deletes the branch) so repeated runs
|
||||||
|
do not silt up the sandbox.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.envfile import load_env_file
|
||||||
|
from nisse_dev.git import GitError, GitRepo, authenticated_url
|
||||||
|
from nisse_dev.gitea import GiteaClient, pull_request_reconciler
|
||||||
|
from nisse_dev.state import Intent
|
||||||
|
|
||||||
|
load_env_file(Path(__file__).resolve().parents[2] / ".env")
|
||||||
|
|
||||||
|
SENTINEL = "NISSE_GITEA_URL"
|
||||||
|
ALSO_REQUIRED = ("NISSE_GITEA_TOKEN", "NISSE_GITEA_TEST_REPO")
|
||||||
|
|
||||||
|
pytestmark = [
|
||||||
|
pytest.mark.integration,
|
||||||
|
pytest.mark.skipif(
|
||||||
|
not os.environ.get(SENTINEL),
|
||||||
|
reason=f"no live Gitea configured ({SENTINEL} unset)",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def intent_for(branch: str) -> Intent:
|
||||||
|
return Intent(
|
||||||
|
id=1,
|
||||||
|
kind="open_pr",
|
||||||
|
payload={"branch": branch},
|
||||||
|
status="pending",
|
||||||
|
created_at="t",
|
||||||
|
resolved_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_open_pr_and_reconcile(tmp_path: Path) -> None:
|
||||||
|
missing = [var for var in ALSO_REQUIRED if not os.environ.get(var)]
|
||||||
|
assert not missing, (
|
||||||
|
f"{SENTINEL} is set, so the live Gitea path is expected to work, "
|
||||||
|
f"but these are missing: {', '.join(missing)}"
|
||||||
|
)
|
||||||
|
base_url = os.environ[SENTINEL]
|
||||||
|
token = os.environ["NISSE_GITEA_TOKEN"]
|
||||||
|
repo_path = os.environ["NISSE_GITEA_TEST_REPO"]
|
||||||
|
client = GiteaClient.from_env()
|
||||||
|
|
||||||
|
account = client.whoami()
|
||||||
|
assert "testing" in account, (
|
||||||
|
f"live tests must run as a testing account, not {account}"
|
||||||
|
)
|
||||||
|
|
||||||
|
branch = f"live-test-{uuid.uuid4().hex[:8]}"
|
||||||
|
reconcile = pull_request_reconciler(client, repo_path)
|
||||||
|
|
||||||
|
# Before the PR exists, external truth says the effect has not happened.
|
||||||
|
assert reconcile(intent_for(branch)) is False
|
||||||
|
|
||||||
|
repo = GitRepo(tmp_path / "work")
|
||||||
|
repo.init()
|
||||||
|
repo.configure_identity(account, f"{account}@example.test")
|
||||||
|
(repo.root / f"{branch}.txt").write_text("live test artifact\n")
|
||||||
|
assert repo.commit_all("Live test commit") is not None
|
||||||
|
|
||||||
|
url = authenticated_url(base_url, repo_path, account, token)
|
||||||
|
pr = None
|
||||||
|
try:
|
||||||
|
repo.push(url, branch, token=token)
|
||||||
|
pr = client.open_pull_request(
|
||||||
|
repo_path,
|
||||||
|
head=branch,
|
||||||
|
base="main",
|
||||||
|
title=f"live test {branch}",
|
||||||
|
body="Opened by the nisse-dev live test suite.",
|
||||||
|
)
|
||||||
|
assert pr.head == branch
|
||||||
|
|
||||||
|
client.post_comment(repo_path, pr.number, "live test comment")
|
||||||
|
bodies = [c.body for c in client.list_comments(repo_path, pr.number)]
|
||||||
|
assert "live test comment" in bodies
|
||||||
|
|
||||||
|
# Now the same reconciler must see the effect and refuse to repeat it.
|
||||||
|
assert reconcile(intent_for(branch)) is True
|
||||||
|
|
||||||
|
# A4 is only real if the server enforces it. The agent's own token
|
||||||
|
# must not be able to merge the PR it just opened, and must not be
|
||||||
|
# able to push to the protected default branch at all.
|
||||||
|
merged = client._http.post(
|
||||||
|
f"/repos/{repo_path}/pulls/{pr.number}/merge", json={"Do": "merge"}
|
||||||
|
)
|
||||||
|
assert merged.status_code >= 400, (
|
||||||
|
"the agent merged its own PR: branch protection is not enforcing "
|
||||||
|
"the human gate"
|
||||||
|
)
|
||||||
|
with pytest.raises(GitError):
|
||||||
|
repo.push(url, "main", token=token)
|
||||||
|
finally:
|
||||||
|
if pr is not None:
|
||||||
|
client.close_pull_request(repo_path, pr.number)
|
||||||
|
repo.delete_remote_branch(url, branch, token=token)
|
||||||
|
client.close()
|
||||||
@@ -33,6 +33,27 @@ pytestmark = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _assert_testing_identity(transport: MatrixTransport, config: MatrixConfig) -> None:
|
||||||
|
"""Two invariants that only a live homeserver can check.
|
||||||
|
|
||||||
|
The configured user id must be the token's real owner, because the
|
||||||
|
transport filters its own messages by comparing sender to that value —
|
||||||
|
a mismatch would make the bot read its own posts as operator input.
|
||||||
|
And tests must run as a testing identity, so a stray live run can never
|
||||||
|
act as the production agent.
|
||||||
|
"""
|
||||||
|
whoami = transport._http.get("/account/whoami")
|
||||||
|
whoami.raise_for_status()
|
||||||
|
actual = whoami.json()["user_id"]
|
||||||
|
assert actual == config.user_id, (
|
||||||
|
f"NISSE_MATRIX_USER_ID is {config.user_id} but the token belongs to "
|
||||||
|
f"{actual}; the self-message filter would not match"
|
||||||
|
)
|
||||||
|
assert "testing" in actual, (
|
||||||
|
f"live tests must run as a testing identity, not {actual}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_send_lands_in_test_room() -> None:
|
def test_send_lands_in_test_room() -> None:
|
||||||
missing = [var for var in ALSO_REQUIRED if not os.environ.get(var)]
|
missing = [var for var in ALSO_REQUIRED if not os.environ.get(var)]
|
||||||
assert not missing, (
|
assert not missing, (
|
||||||
@@ -43,6 +64,7 @@ def test_send_lands_in_test_room() -> None:
|
|||||||
marker = f"transport smoke test {uuid.uuid4().hex}"
|
marker = f"transport smoke test {uuid.uuid4().hex}"
|
||||||
transport = MatrixTransport(config)
|
transport = MatrixTransport(config)
|
||||||
try:
|
try:
|
||||||
|
_assert_testing_identity(transport, config)
|
||||||
transport.send_message(marker)
|
transport.send_message(marker)
|
||||||
response = transport._http.get(
|
response = transport._http.get(
|
||||||
f"/rooms/{config.room_id}/messages",
|
f"/rooms/{config.room_id}/messages",
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
"""End-to-end milestone against real models: plan with the planner role,
|
||||||
|
approve through the gate, then let the executor role actually create and test
|
||||||
|
a file on disk.
|
||||||
|
|
||||||
|
Gated on NISSE_LIVE_PLANNER_MODEL like the planning test — it needs a model
|
||||||
|
that holds the tool-call contract on both ends.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.control import (
|
||||||
|
AWAIT_ARCH_SIGNOFF,
|
||||||
|
AWAIT_INPUT,
|
||||||
|
AWAIT_MILESTONE_SIGNOFF,
|
||||||
|
EXECUTING,
|
||||||
|
Orchestrator,
|
||||||
|
)
|
||||||
|
from nisse_dev.envfile import load_env_file
|
||||||
|
from nisse_dev.executor import LLMExecutor
|
||||||
|
from nisse_dev.llm import LiteLLMClient, RoleMap
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.planner import LLMPlanningAgent
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import Workspace
|
||||||
|
from nisse_dev.transport import InMemoryTransport
|
||||||
|
|
||||||
|
load_env_file(Path(__file__).resolve().parents[2] / ".env")
|
||||||
|
|
||||||
|
SENTINEL = "NISSE_LIVE_PLANNER_MODEL"
|
||||||
|
|
||||||
|
pytestmark = [
|
||||||
|
pytest.mark.integration,
|
||||||
|
pytest.mark.skipif(
|
||||||
|
not os.environ.get(SENTINEL),
|
||||||
|
reason=f"no live planner route configured ({SENTINEL} unset)",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _drive_to_execution(
|
||||||
|
store: StateStore, transport: InMemoryTransport, max_steps: int = 10
|
||||||
|
) -> None:
|
||||||
|
"""Play the operator. A scripted approve/approve/start sequence is not
|
||||||
|
robust against a live model: ask_human is a legitimate move at any gate,
|
||||||
|
and a question shifts the state machine under a fixed script. So respond
|
||||||
|
to whatever the agent actually did."""
|
||||||
|
for _ in range(max_steps):
|
||||||
|
state = store.get_state()
|
||||||
|
assert state is not None
|
||||||
|
if state.state == EXECUTING:
|
||||||
|
return
|
||||||
|
gate = state.data.get("gate") or {}
|
||||||
|
if state.state == AWAIT_INPUT:
|
||||||
|
transport.receive("don", "Use your best judgement and proceed.")
|
||||||
|
elif state.state == AWAIT_MILESTONE_SIGNOFF and gate.get("approved"):
|
||||||
|
transport.receive("don", "start milestone 1")
|
||||||
|
elif state.state in (AWAIT_ARCH_SIGNOFF, AWAIT_MILESTONE_SIGNOFF):
|
||||||
|
transport.receive("don", "approve")
|
||||||
|
else:
|
||||||
|
raise AssertionError(f"unexpected state {state.state}")
|
||||||
|
raise AssertionError(
|
||||||
|
f"never reached execution; last message: {transport.outbound[-1]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_plan_approve_execute_produces_a_real_file(tmp_path: Path) -> None:
|
||||||
|
assert os.environ.get("NISSE_LLM_TOKEN"), (
|
||||||
|
f"{SENTINEL} is set, so the live path is expected to work, "
|
||||||
|
"but NISSE_LLM_TOKEN is missing"
|
||||||
|
)
|
||||||
|
# Both roles use the capable model here: the point is the whole gated
|
||||||
|
# loop over a real proxy, not local-model competence (measured separately).
|
||||||
|
model = os.environ[SENTINEL]
|
||||||
|
roles = RoleMap(executor=model, planner=model, delegate=model, consultant=model)
|
||||||
|
client = LiteLLMClient.from_env()
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
workspace = Workspace(tmp_path / "work", timeout_s=60)
|
||||||
|
transport = InMemoryTransport()
|
||||||
|
orchestrator = Orchestrator(
|
||||||
|
store,
|
||||||
|
transport,
|
||||||
|
LLMPlanningAgent(client, store, roles),
|
||||||
|
operators={"don"},
|
||||||
|
roles=roles,
|
||||||
|
executor=LLMExecutor(client, store, roles, workspace, max_turns=10),
|
||||||
|
)
|
||||||
|
orchestrator.start()
|
||||||
|
|
||||||
|
transport.receive(
|
||||||
|
"don",
|
||||||
|
"use greet. Milestone 1: create greet.py in the workspace root defining "
|
||||||
|
"greet(name) returning 'Hello, <name>!', and verify it by running python.",
|
||||||
|
)
|
||||||
|
_drive_to_execution(store, transport)
|
||||||
|
|
||||||
|
state = store.get_state()
|
||||||
|
assert state is not None
|
||||||
|
assert state.state in (EXECUTING, AWAIT_INPUT)
|
||||||
|
|
||||||
|
created = list(workspace.root.rglob("*.py"))
|
||||||
|
assert created, f"executor created no Python file; said: {transport.outbound[-1]}"
|
||||||
|
combined = "\n".join(p.read_text() for p in created)
|
||||||
|
assert "def greet" in combined
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
"""Live Anthropic passthrough: the vendor-native route delegations will use.
|
||||||
|
|
||||||
|
Proves the renderer's body is accepted as-is, that a tool call round-trips
|
||||||
|
through the canonical log, and — per A5 — that prompt caching survives the
|
||||||
|
passthrough rather than being flattened by the proxy.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from nisse_dev.canonical import AssistantText, ToolCall, ToolResult, UserMessage
|
||||||
|
from nisse_dev.envfile import load_env_file
|
||||||
|
from nisse_dev.llm import AnthropicPassthroughClient
|
||||||
|
from nisse_dev.oplog import OpLog
|
||||||
|
from nisse_dev.state import StateStore
|
||||||
|
from nisse_dev.tools import EXECUTOR_TOOLS
|
||||||
|
|
||||||
|
load_env_file(Path(__file__).resolve().parents[2] / ".env")
|
||||||
|
|
||||||
|
SENTINEL = "NISSE_LIVE_PLANNER_MODEL"
|
||||||
|
|
||||||
|
pytestmark = [
|
||||||
|
pytest.mark.integration,
|
||||||
|
pytest.mark.skipif(
|
||||||
|
not os.environ.get(SENTINEL),
|
||||||
|
reason=f"no live frontier route configured ({SENTINEL} unset)",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def store_with(tmp_path: Path, *events) -> StateStore:
|
||||||
|
store = StateStore(tmp_path / "state.db", oplog=OpLog("nisse-dev", io.StringIO()))
|
||||||
|
for event in events:
|
||||||
|
store.append_event(event)
|
||||||
|
return store
|
||||||
|
|
||||||
|
|
||||||
|
def test_passthrough_completes_a_plain_turn(tmp_path: Path) -> None:
|
||||||
|
store = store_with(
|
||||||
|
tmp_path, UserMessage(text="Reply with exactly: pong", source="live")
|
||||||
|
)
|
||||||
|
result = AnthropicPassthroughClient.from_env().complete(
|
||||||
|
store.read_events(), None, os.environ[SENTINEL], system="Be terse."
|
||||||
|
)
|
||||||
|
texts = [e.text for e in result if isinstance(e, AssistantText)]
|
||||||
|
assert texts and "pong" in texts[0].lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_passthrough_round_trips_a_tool_call(tmp_path: Path) -> None:
|
||||||
|
store = store_with(
|
||||||
|
tmp_path,
|
||||||
|
UserMessage(
|
||||||
|
text="List the files in the current directory using your tools.",
|
||||||
|
source="live",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
client = AnthropicPassthroughClient.from_env()
|
||||||
|
produced = client.complete(
|
||||||
|
store.read_events(),
|
||||||
|
EXECUTOR_TOOLS,
|
||||||
|
os.environ[SENTINEL],
|
||||||
|
system="Use a tool. Do not answer in prose.",
|
||||||
|
)
|
||||||
|
calls = [e for e in produced if isinstance(e, ToolCall)]
|
||||||
|
assert calls, f"expected a tool call, got {produced!r}"
|
||||||
|
|
||||||
|
# Feed the result back: the follow-up render must be accepted, which is
|
||||||
|
# what proves tool_result placement is right for this vendor.
|
||||||
|
for event in produced:
|
||||||
|
store.append_event(event)
|
||||||
|
store.append_event(
|
||||||
|
ToolResult(call_id=calls[0].call_id, result="app.py\ntests/", success=True)
|
||||||
|
)
|
||||||
|
follow_up = client.complete(
|
||||||
|
store.read_events(), EXECUTOR_TOOLS, os.environ[SENTINEL]
|
||||||
|
)
|
||||||
|
assert follow_up, "the vendor rejected or ignored the tool_result turn"
|
||||||
|
|
||||||
|
|
||||||
|
def test_prompt_caching_survives_the_passthrough() -> None:
|
||||||
|
# A5 asks for this to be verified rather than assumed: a proxy that
|
||||||
|
# flattened the request would silently drop cache control and turn a
|
||||||
|
# cost optimisation into a per-call expense.
|
||||||
|
base = os.environ["NISSE_LLM_BASE_URL"].rstrip("/")
|
||||||
|
token = os.environ["NISSE_LLM_TOKEN"]
|
||||||
|
response = httpx.post(
|
||||||
|
f"{base}/v1/messages",
|
||||||
|
headers={
|
||||||
|
"Authorization": f"Bearer {token}",
|
||||||
|
"x-api-key": token,
|
||||||
|
"anthropic-version": "2023-06-01",
|
||||||
|
},
|
||||||
|
json={
|
||||||
|
"model": os.environ[SENTINEL],
|
||||||
|
"max_tokens": 16,
|
||||||
|
"messages": [{"role": "user", "content": "Say pong."}],
|
||||||
|
},
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
usage = response.json().get("usage", {})
|
||||||
|
assert "cache_read_input_tokens" in usage, (
|
||||||
|
f"cache accounting absent from passthrough usage: {usage}"
|
||||||
|
)
|
||||||
Reference in new issue
Block a user