Commit Graph
14 Commits
Author SHA1 Message Date
dongarandClaude Opus 5.5 6ae2375194 M2.8: switch over to the new loop
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m16s
CI / e2e (pull_request) Successful in 10m51s
nisse-dev now runs only the conversational loop. The gated state machine
it was built alongside is gone:

- Removed control.py, planner.py, executor.py, approvals.py, gates.py
  (command_form moves to loop/reader.py), publish.py and poller.py, the
  old assembly in agent.py (its shared helpers move to loop/build.py),
  the store's state-machine view (get_state, transition), their tests,
  the old end-to-end and live planning/milestone/gate tests, and
  specs/orchestrator.qnt with its scenarios.
- Roles are driver, helper, expert and classifier; driver is required.
  loop:, roles.planner, roles.executor and tuning.executor are refused
  with what to use instead; tuning.driver takes the turn cap and failure
  limit.
- The benchmark and the capability matrix measure the driver: the
  loop's own system prompt and tools, the task's plan as the agreed
  restatement. Task plans end with finish_milestone, so earlier runs
  show as (old); stored runs' executor tuning reads as the driver's.
- A delegation on the loop records what its helper's alias resolved to
  (L9), as the old path's did.
- The dump shows a work item's revision and what it waits for.
- Behaviour the old tests covered and the loop shares stays covered:
  driver runs that fail on a model call, stop mid-turn, reset the
  failure count; go-aheads ignoring case and punctuation; a request too
  big to fit is never sent; the assembly tests, now on build_loop_agent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-08 03:12:58 +00:00
dongarandClaude Opus 5.5 e35aea5045 Test levels: smoke e2e on PRs, the full set on e2e:full and before a release
- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16
  property walks from a seed that moves with the run number, integration
  tests, and the end-to-end smoke tests (the slice, a conversation filed
  as work, a question; `smoke` marker).
- A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and
  the whole end-to-end suite. Adding e2e:full to a PR runs it; any other
  label runs nothing.
- A published release runs the full set before its image is built.
- The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0
  by default).

Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes
serially.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-07 22:29:41 +00:00
dongarandClaude Opus 5.5 8994b3fb4b Conversation on the new loop (M2.7)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 13m26s
CI / e2e (pull_request) Successful in 38m44s
- Questions never change state (L5, #77): a message ending in "?" to a
  waiting project item starts a question run (reply and look-only tools,
  an instruction to change nothing), and the item goes back to the very
  wait it was in: a restatement up stays up and a go binds to it; the
  agent's own question stays open.
- Explanations: a restatement gives its reasons and the main alternative;
  a revision says what changed (restate(changed), sent back without it);
  a milestone's summary says what was done, verified and left open.
- Progress in prose (#42): the driver's and a helper's own words for a
  turn are posted, trimmed, at most once a minute per item; `details`
  shows the last tool calls; a message arriving mid-run is acknowledged at
  once ("Noted; ...").
- CI: the e2e job's comments and the plan no longer claim a newer push
  cancels the run it supersedes (Gitea here doesn't); the concurrency
  block goes.

Tests: conversation unit tests (questions in a go-ahead and an answer wait,
an unanswerable question, revisions, progress rate and trimming, details,
mid-run notes), the helper's progress, the walk asking questions in every
state with a questions_change_nothing property (routing questions the old
way fails 32 of 48 seeds), and an e2e question while a restatement is up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-07 18:01:51 +00:00
dongarandClaude Opus 5.5 ad2d090143 New projects from conversation (M2.5, PR B)
- propose_project(name, purpose, decisions) in a conversation goes up for
  a go-ahead (checked: a slug, not an existing project, no trailing
  question).
- On go, the provisioning checklist (D1): a repository with a README,
  protection on its default branch, the agent as a collaborator, a build
  VM, a deploy key, and the project entry as YAML with <...> to fill in.
- On "done", the agent reads its configuration again and adds the new
  project live (build_loop_agent(load=...); only additions), then checks
  what was provided (D4, verified, not assumed): permissions and
  protection as it sees them, then on a throwaway branch with an empty
  commit, a branch push must succeed and a push to the default branch and
  merging its own PR must be refused. The check's PR is closed and its
  branch deleted. Failures are said per item; nothing starts.
- Checked: the tracking issue (the distilled decisions, carrying the new
  item's ID), and the project's item, agreed and labelled `design`, whose
  first milestone is ARCHITECTURE.md and the build plan. A conversation
  waiting on provisioning doesn't close as idle.
- The walk replays exactly from its seed (item IDs too), and event IDs
  stay unique across simulated restarts, as a server's are.
- The test config holds output to 8192 tokens, under the test key's
  per-request cost cap.

Tests: new-project unit tests (checklist and its YAML parse as config, done
before the entry, what isn't set up, a checked project's record and first
PR, an existing name), protection enforced by a hook on the fake forge,
the walk proposing projects and saying done, and an e2e new project in a
repository the test operator provisions from the agent's checklist.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-07 15:39:00 +00:00
dongarandClaude Opus 5.5 6db80dfb4b End-to-end tests on a local model, and in CI on every PR
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 5m26s
CI / e2e (pull_request) Successful in 17m27s
- Every e2e role runs on qwen3.8-27b on the RTX 4090 (e2e_roles();
  NISSE_E2E_MODEL overrides, e.g. claude-haiku-4-5). The first full run
  passed 7 of 8 in 26 minutes for $0.0000; the eighth was a real finding,
  fixed with the new-project PR.
- scripts/e2e.sh reports a run's spend on the test key and the day's total
  against its budget: above $0 on the local model means a paid fallback.
- CI runs the suite as its own job on pull requests (not pushes to main),
  on a throwaway sshd (--local), cancelling a run a newer push supersedes,
  so the checks job still reports in minutes.
- The plan says so: e2e on every PR; locally only -k / --lf.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-07 15:14:43 +00:00
dongarandClaude Opus 5.5 679ace1fd8 A repository per test run instead of the shared sandbox
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 5m17s
Every live test that touches a repository gets one of its own from
e2e_support.fresh_repo, made by the test operator as an operator makes a
project's (D4, D8): seeded from tests/integration/seed/ (the sandbox's
package, without past runs' leftovers), the agent's account a collaborator
with write access, main protected (pushes only by the owner, an approval to
merge), and a deploy key generated for the run. It is deleted afterwards
(NISSE_E2E_KEEP_REPOS keeps it); scripts/e2e-cleanup.sh deletes leftovers.

- Both e2e fixtures use it; per-run base branches are gone. The merge
  helper approves first, as the protection now requires.
- test_gitea_live uses it, so CI needs NISSE_OPERATOR_TOKEN and no longer
  writes the sandbox deploy key (NISSE_PUSH_KEY).
- A live test of the helper: seeded, shared with the agent, protected
  against it, deleted.
- nisse-test.yaml's project entry is a placeholder; pytest and ruff skip
  the seed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-06 05:10:27 +00:00
dongarandClaude Opus 5.5 3f54896f65 Conversations on the new loop (M2.5, PR A)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 5m14s
- A top-level message that isn't a command is a conversation: an item in
  the `general` scope (L1), threaded under the operator's message, with
  `reply`, `ask_human` and `propose_work` and no workspace or execution.
  `general` is reserved as a project name.
- propose_work(project, title, summary) goes up as a revision for a
  go-ahead (or a plain yes). On go, under a file_issue intent (L8), the
  record issue is opened in that project, assigned to the agent and
  carrying the new item's ID; its item starts agreed, with the summary as
  its brief. Intake skips any issue carrying an ID (D1); boot adopts a
  filed issue or puts the proposal up again.
- A conversation closes after tuning.loop.general_idle_s (default a day);
  a reply in its thread then starts a new conversation that continues it.

Tests: conversation unit tests (threading, filing, the marker, an unknown
project, changes, idle close and continuation, a plain yes, both crash
cases, commands at top level); the walk with conversations, proposals and
idle time, and records_have_one_item / conversations_do_no_work; an e2e
conversation that becomes filed work.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-06 04:26:40 +00:00
dongarandClaude Opus 5.5 064d9bf554 Projects as configuration entries (M1.4, D8)
CI / checks (push) Skipped
CI / checks (pull_request) Failing after 5m7s
- Config: `projects: {name: entry}` with repo, base_branch, workspace,
  build_host, push credential, labels (defaults added to every issue),
  secrets by ${VAR} reference with visibility (commands | harness), and
  egress (recorded, not enforced). Every problem in every entry is
  reported at once; an inline secret, a bad visibility, a repeated repo or
  a non-slug name is an error.
- The old loop runs exactly one project (only_project) and refuses more.
- The new loop is per project: each its own clone, workspace (with its
  commands-visible secrets, redacted on return), push credential and
  milestones; intake polls every project's repository; items are scoped
  to their project for every run, publish and PR watch. Issues are named
  project#14 when there is more than one project, and a bare #14 that
  matches two is asked about. Switching projects parks the one worked on
  last (all others after a boot).
- A restatement that ends in a question is sent back to the driver, not
  posted: the e2e driver kept asking "Is this the right interpretation?",
  which a plain yes can't answer.

Tests: config entries and errors, two-project assembly, two-project loop
tests (intake, own workspaces and repos, #14 in both, do-next, label
defaults, parking across projects), the property walk over two projects
with a new own_repository_only property (an item using the first
project's repo fails 15 of 16 seeds), and the restatement check.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-06 01:12:57 +00:00
dongarandClaude Opus 5.5 cd9ab2559e Multi-milestone work and documents on the new loop (M2.4)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 3m45s
- Every milestone branch is cut from the latest default branch: the next
  milestone builds on the merged previous one, and a redo after a closed PR
  drops the rejected work. Uncommitted work comes along.
- While the item's PR is open, start_milestone is refused, with the PR's
  state read from Gitea at that moment, not the last watch round.
- finish_milestone(final): earlier PRs say "Part of #N", the last "Closes
  #N"; unsaid, it's final, as before.
- close_item with the item's PR still open waits for the merge (L2: a
  closed item leaves nothing open).
- The driver's instructions cover ARCHITECTURE.md and build plans under
  docs/plans/ (D2); the harness never parses a plan.
- Parking stashes uncommitted work and pushes it to <item>-parked, put back
  on resume, instead of committing it onto the item's branch. The walk
  found a half-done review fix parked onto a PR that had merged meanwhile,
  where it was lost. Architecture D4's parking line says so.

Tests: unit tests for fresh milestone branches, the merge wait (including
a merge not yet watched), redo after a closed PR, Part of/Closes, close
waiting on an open PR, and the parked-fix regression; the walk's new
milestones_start_fresh property (fork-from-current fails 9 seeds); an e2e
two-milestone feature.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-05 20:57:37 +00:00
dongarandClaude Opus 5.5 4d9c1d7529 Threads, commands and labels on the new loop (M2.3)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 3m45s
- Commands that name their item ("go #14", "abandon #14", "stop #14")
  apply to it wherever they're said; go, approve and abandon alone at top
  level are asked about (L5). stop is never taken as an answer.
- A plain-language reply to a restatement is read by GoAheadReader, one
  model call that agrees only on a clear yes; questions never reach it. The
  agent says "taking that as a go-ahead on rev-N" first, bound to the
  revision on screen at arrival (L3, L5, D1).
- Labels steer (D1): `bug` goes ahead after tuning.loop.bug_grace_s
  (default 600 s) unless the operator replies; `discuss` isn't offered
  restate until the operator has spoken; `bug`, `discuss` and `design` add
  to the driver's instructions. A restatement asks no questions.

Tests: label and command unit tests, the property walk with labelled
issues, plain yeses, named gos and time passing (grace_only_unanswered,
discuss_talks_first), and e2e tests for a bug proceeding after its grace
period and a plain-language go-ahead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-04 16:13:22 +00:00
dongarandClaude Opus 5.5 cceb30c729 The queue: parking, the operator's order, blocked items (M2.2)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 3m26s
Several work items on the new loop (L1, D4):
- Switching parks the item on the workspace: its work is committed and its
  branch pushed; resuming checks the item's branch back out. Branches of
  open items are never set aside by a fresh workspace.
- Next item: the operator's "do #N next" first, otherwise the oldest.
- Issues with open Gitea dependencies are blocked until they close, checked
  at intake and on every round; status shows blocked items and the line.

Tests: queue unit tests, the property walk over several issues with
dependencies and a new parked_work_safe property (it catches parking
disabled), and an end-to-end test with two issues where B depends on A.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-04 13:34:09 +00:00
dongarandClaude Opus 5.5 651551aab4 Answer where asked, wait on a merge, and let the driver title PRs
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m17s
From the new loop's first live run (sandbox #255, PR #256):
- A question asked on the PR is answered on the PR (an inline one quoted
  with its file and line, since the API can't reply in its thread), with
  a pointer in the item's Matrix thread; one asked in Matrix stays there.
- wait_for_review: when a review needs no change, or only the operator's
  merge is left, the item waits on the merge instead of on an answer.
- finish_milestone takes the PR's title from the driver.
The e2e test's merge waits out Gitea's brief 'try again later' after a
push.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-03 17:29:28 +00:00
dongarandClaude Opus 5.5 37965aefca Fix the run-breaking tool-result order; e2e test runs the slice (M2.1, PR D)
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m17s
The first live run of the new loop's e2e test found that a fact recorded
while start_milestone was in flight landed between the tool call and its
result; both API formats reject that, so every later request in the run
failed. Views now keep each result right after its call. Scripted test
models check the APIs' ordering rules, so this can't pass unit tests
again.

The e2e tests no longer depend on the sandbox's history: each run asks for
a uniquely named function and works against a base branch of its own,
deleted afterwards. The review step asks for a change that can't already
be there. The scripted operator gives up after five fruitless answers, and
each run keeps its state store for inspection. The driver's prompt says
review fixes stay in the current milestone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-03 15:11:06 +00:00
dongarandClaude Opus 5.5 897181b0f4 Add the end-to-end test of the new loop's slice (M2.1, PR D part 2)
CI / checks (push) Skipped
From an issue the scripted operator files and assigns in Gitea, through
the restatement, go-ahead, milestone and PR, a review comment answered
with a pushed fix, to an approved and merged PR and a closed item and
issue. The operator acts in Gitea through its own test account
(NISSE_OPERATOR_TOKEN) and in the item's thread through the in-memory
transport. Runs from scripts/e2e.sh.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-10-03 14:20:07 +00:00