Five runs per task through LiteLLM on the RTX 4090. qwen3.8-27b (thinking)
passed 36 of 40. Flash-Next was run thinking, at reasoning_effort low, and
with no thinking.
The first Flash-Next runs used non-git workspaces, where `git log` fails,
so their task hashes are older and the report marks them (old). On git
workspaces (#177), both thinking and lowthink passed 32 of 40; nothink
passed 12 of 40 on the old workspaces and was not rerun. qwen3.8-27b had no
git-caused failures, so it was not rerun.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
nisse-core records each model call of every agent into a file beside its
state store (<state>.calls.db), for the status page (L7, X6):
- the request body exactly as sent (never headers), the raw response with
its reasoning, the model asked and the one that answered, time, tokens
and llama-server's own prompt reuse (timings.cache_n);
- its work item, role, run and purpose: Driver.run and nisse-dev's
delegation loop open runs, nested; compaction, expert consults, the
classifier and the go-ahead reader label their calls;
- provenance, worked out in BudgetedClient by comparing the events at
each stage: each event sent with its log id and content as sent, which
a compaction replaced, which fitting shortened (by how much) and
dropped.
Content is stored once by hash, compressed. It is diagnostic only: the
agent never reads it, a failed write is logged once and recording stops
while the call goes on, and the file is deleted when the agent starts,
so its layout never migrates; CallReader refuses a layout it doesn't
know. While recording, the system prompt ends with one sentence saying
so. `call_record: false` in either agent's YAML turns it off.
The benchmark records each run beside that run's own state store, and
`--keep-runs DIR` keeps both for the page.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
A run that repeats itself, or keeps getting the same correction, ends in
code as escalated instead of running to its turn cap (L2, M7.8), and the
page flags such runs (X6). While the call record is on, each system
prompt says so.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
nisse-ui (X6) shows the operator each agent's work and, per item, a
walkthrough down to every model request and response, for studying how
context, compaction and memory work. It reads each agent's state store,
call record and memory folder through read-only mounts, so agent code is
unchanged and each agent stays the only writer of its state; it is
released with the agents and tested against stores their own code wrote.
The call record (L7) is new in nisse-core: every request as sent, the raw
response with its reasoning, timing, tokens, prompt reuse and each
message's provenance. It is diagnostic only, never read by the agent, and
deleted at start, so it never migrates.
Commands from the page, including a hold on a task or a whole agent, are
an open question (Part Q). The shared database first proposed in #109 is
dropped. The plan gets M6.6.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
The agent works in a clone, but benchmark workspaces were plain
directories, so a model that starts with `git log` failed two commands
in a row and was stopped as stuck within seconds (Flash-Next, 10 of 13
failures on 2026-10-24). The seed is now committed on main, locally and
on the build VM, and how a workspace is set up is part of each task's
hash, so earlier runs are reported as not comparable.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Each call through BudgetedClient emits llm.call: seconds, input,
output and cached tokens, and the answering model when a fallback
answered; a failed call logs how long it ran before it failed.
Usage now carries the server's cached-prompt count on both routes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
A `review` target in the Dockerfile: its own user (nisse-review, uid
10002), git for its clones, state and clones in /var/lib/nisse-review,
configuration at /etc/nisse-review, and only its own agent installed.
release.yml publishes gitea.dongar.net/dev/nisse-review beside nisse-dev,
at the same version. review-bringup.sh builds and runs the image, as
bringup.sh runs nisse-dev's.
The test stage adds an agent package its base lacks, without its
dependencies, since the suite's own code imports every agent's. Without
that, the tests over nisse-dev's image no longer import since
test_review_live arrived. tests/deploy/test_images.py checks the image and
that the test stage leaves what it installed as it was.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
nisse-review's review of #169: a reply quoting a top-level point, or one
Gitea refused on its line, arrives through list_comments, so answering it
would start the thank-you loop again. Also records what the live Gitea
showed: resolving is per comment, and resolving again is a 204.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Gitea now has two runners but there is one 4090. The e2e job joins one
concurrency group without cancel-in-progress, so a second PR's e2e waits
for the first. checks still run in parallel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
The reviewer is read-only (R3), so Gitea refuses it resolving a thread.
Now each reply says whether it settles its thread, and a settled reply
carries <!-- nisse-review: settled -->. nisse-dev, as the author, resolves
every comment on that thread's line when it watches the PR, and doesn't
take the settled reply as a point to answer. scripts/resolve_settled.py
does the same for PRs made by hand.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
On #166 the reviewer called post_review with two points unanswered, was
refused, answered them, then only skipped already-answered points until the
40-turn cap: the review went out as partial, its summary lost. A refused
ending call is now held, and the driver ends the run with it as soon as the
last point is settled (a new due hook in nisse-core's driver; nisse-dev
doesn't use it). A held call that is wrong for another reason by then
(sign_off over drafted comments) is left to the model. With nothing open,
reply and skip say which tool ends the run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
The operator chose single-space mode: the vault is mounted at /data, with
memory/ read-only over it, and there are no accounts or setup wizard;
Authelia in the operator's Caddy is the login. SB_NAME forces the mode, so
an empty vault can't boot the wizard. SilverBullet's own data folder is
gone. check now fails if the wizard could be open. The README says how to
make memory/ writable for a while and back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
SB_NAME alone put SilverBullet 2.11 in single-space mode: no accounts, no
setup wizard, and its /data folder served instead of the vault. The compose
file sets only PUID and PGID now, a test holds it to that, and the host
check fails on a single-space boot.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
The check went to the host on stdin, and the first docker compose exec read
the rest of it: the memory and firewall checks never ran, and it passed.
It is now its own file, sent as an argument, and ends with "all checks
passed" only when every check has run. Tests fake the host with a docker
that reads stdin.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Each agent ships as its own image, since that's how it runs on the agent
host. nisse-review's image becomes M7.6, ahead of the agent host; its
deployment moves to M7.7. nisse-ops' image comes with its first slice.
Also fixes two stale references to M7.6.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
deploy/agent-host sets up the vault on nisse-host (infra/tofu#77) and
SilverBullet over it: the operator writes the vault, the agents' memory/
is read-only to it, and only the operator's Caddy reaches it, which
serves vault.dongar.net to the LAN. scripts/agent-host.sh installs and
checks it from here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Opus's provider filter declined one of nisse-review's ask_expert
questions (which repositories a Gitea token can list) every time. It came
back as a 200 with no text, and the expert reported "unparseable reply:
empty reply". Both LLM clients now raise ModelRefused for
finish_reason content_filter (chat route) and stop_reason refusal
(passthrough). The expert's handback says what happened and what to do
instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Gitea dismisses an approval made stale by a push in a job it queues on the
push (services/pull AddTestPullRequestTask, v28.1.0), and its merge check
counts approvals not yet dismissed. Measured here, the approval stands for
0.15-0.23 s after the push; the test's merge came a couple of API calls
later, so it usually lost the race, and once won it. The test now waits for
the dismissal, with a timeout, before expecting Gitea's refusal, and checks
the agent's own rule at once: the PR's head is read from the branch, so the
approval is for an older commit straight away.
Also notes in the Development Plan why the helper loop doesn't compact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Before each driver call, over compact_at (0.6) of the prompt budget, the
same model is asked for notes over the very request it was about to get,
plus one message, so the server's cache covers all but the notes. The notes,
with code's list of the calls they replace, become a compaction event that
renders in place of its range, keeping word for word what started the work
and the last few turns. Rare, large steps keep the prompt's start unchanged
between them. A later compaction folds in the earlier; one from an earlier
run's view is hidden.
Failed notes are logged and fitting takes over. Fitting now applies
compactions before it measures, records what it cuts (Trim) and logs each
cut as context.trimmed. Wired into both agents' drivers and the benchmark;
nisse-dev's helper loop is a follow-up.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
Archives were downloaded and unpacked afresh for every run, which doesn't
scale to very large repositories. Each repository is now cloned once, with
its whole history, under scratch/repos/<owner>/<name>/ (head/ the clone,
base/ a worktree when read). A run fetches every branch and the PR's
refs/pull/N/head, checks out the head exactly, and fails if that head is
gone. Only the latest repository is kept for now.
Git runs walled off: none of the host's git configuration, links checked
out as plain files, no hooks, submodules or LFS, only the Gitea URL's
protocol, and the token as an http.extraHeader for that URL in git's
environment only. A stop ends a git command; a cut-short first clone is
removed. The reader never shows .git. GiteaClient.archive is gone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
SIGTERM used to stop only new rounds: a review in progress ran on until
systemd killed it 90s later. Now the scheduler's stop sets the driver's
cancel flag, so the run ends at its next turn, and a model call in flight
is broken off (Interruptible, raising Stopped from the signal handler, only
ever inside a model call). The item goes back to ready, with nothing
drafted posted and no failure counted; replies already posted carry their
markers and are not repeated.
intake.repo_failed is logged when a repository starts failing or the reason
changes, and intake.repo_ok when it lists again, not every round.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
The property walk (seed 468, a CI run's moving seed) found it: when the
build VM went out of reach while _delegate moved the helper's work back
onto the milestone's branch, git's HostUnavailable came back as a
GitError, was noted as a fact, and the driver's run went on, calling the
model with its VM gone (no_model_on_a_lost_vm). Now a GitError caused by a
lost VM raises VMUnreachable once the delegation's intents are settled, so
the run ends and the VM is blocked, as for any other lost-VM call (#137).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
dev-agent
dongarandClaude Opus 5.5
bot-claudeandClaude Opus 5.5