16 Commits

Include renames

Author	SHA1	Message	Date
dongarandClaude Opus 5.5	064d9bf554	Projects as configuration entries (M1.4, D8) ... - Config: `projects: {name: entry}` with repo, base_branch, workspace, build_host, push credential, labels (defaults added to every issue), secrets by ${VAR} reference with visibility (commands | harness), and egress (recorded, not enforced). Every problem in every entry is reported at once; an inline secret, a bad visibility, a repeated repo or a non-slug name is an error. - The old loop runs exactly one project (only_project) and refuses more. - The new loop is per project: each its own clone, workspace (with its commands-visible secrets, redacted on return), push credential and milestones; intake polls every project's repository; items are scoped to their project for every run, publish and PR watch. Issues are named project#14 when there is more than one project, and a bare #14 that matches two is asked about. Switching projects parks the one worked on last (all others after a boot). - A restatement that ends in a question is sent back to the driver, not posted: the e2e driver kept asking "Is this the right interpretation?", which a plain yes can't answer. Tests: config entries and errors, two-project assembly, two-project loop tests (intake, own workspaces and repos, #14 in both, do-next, label defaults, parking across projects), the property walk over two projects with a new own_repository_only property (an item using the first project's repo fails 15 of 16 seeds), and the restatement check. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-06 01:12:57 +00:00
dongarandClaude Opus 5.5	e02bac110d	Publish only the work: leave build artifacts out and report them (#80) ... The clone gets the standard build artifacts in its own exclude file, so they never show as changes, even to a model's `git add -A`. The harness's commits leave the same patterns out even when the project tracks them, as the sandbox's main did. What was left out is reported in the publish note and the PR description. Found again in bring-up run 3 (sandbox PR #208). Fixes #80. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-02 22:38:13 +00:00
dongarandClaude Opus 5.5	c931ac939b	Let pushes go by an HTTPS token or an SSH deploy key ... The push credential is either a push-only account's token, handed to git through a credential helper that names environment variables, never values, or a deploy key in a short-lived ssh-agent. The remote URL follows the credential. Configured as gitea.push_user and push_token, or gitea.push_key. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-02 21:59:56 +00:00
dongarandClaude Opus 5.5	761ebdb857	Run every command on the build VM over SSH (M1.3) ... execution.SSHHost.execute is now the only way the agent runs anything (L13). Commands travel as a script on SSH's stdin, so the secrets handed to a call are never on a command line, and every value handed out is redacted from the output. Timeouts kill the remote process group, and an unreachable host is reported as such, not as a failed command. Workspace and GitRepo run on the build VM. The clone uses the repository's SSH URL, so no credential is stored in it (#89); fetches and pushes get the push key over stdin, in a short-lived ssh-agent, and PRs are still opened from the agent's host with its API token. The bench runs on the build host too. Tests reach "the build VM" through a throwaway local sshd (nisse_dev .testing); CI installs openssh-server. The live push test now pushes with the sandbox's deploy key (NISSE_PUSH_KEY). Fixes #89. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-02 21:39:46 +00:00
dongarandClaude Opus 5.5	83804fe02a	Put work items in the state store, and bind the log to them (M1.1) ... Development Plan M1.1, as discussed. - Schema v4: a work_items table (id, scope, title, status, outcome, position, data) and an agent row whose active_item points at one (NULL: idle). Canonical events and intents carry their work item, and events their actor. Older stores are refused with a clear message: there was no production data, so no migration (the mechanism stays). - The log is reached through a Transcript bound to one work item and one actor: store.transcript(item, actor) reads only that item and writes with both. The planner, executor, delegation and expert desk get one per call from the orchestrator instead of holding the store, so a component can neither read another task's events (the planner read every earlier task's, #78) nor write without attribution. store.all_events() is kept for readers that span items (evaluation capture, dumps). Intents are begun through the transcript or with work_item=, so each belongs to its item. - The orchestrator opens a work item on 'use', closes it with its outcome (completed or abandoned), and carries what the operator said while idle into the new item as copies with their origin, so a task sent before 'use' is still planned from (D1). control.py keeps its state machine through get_state()/transition(), now over the active work item, until M2.6 retires it. - nisse_dev.dump shows work items and each event's item and actor. Tests: test_work_items.py (scoping, attribution, intent ownership, restart, outcomes, carried messages, a second task never seeing the first); the suites updated for the per-call transcript. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-01 23:20:29 +00:00
dongarandClaude Opus 5.5	3cf4cefacd	Rename the delegate and consultant roles to helper and expert ... Architecture L9's role names, for the two roles with a direct counterpart: the delegation role is now 'helper' and the ask_expert role 'expert', in RoleMap, configuration (roles.helper, roles.expert), recorded events and tests. 'planner' and 'executor' stay distinct for now; they become 'driver' when the new loop merges them (Development Plan M2.3). Configurations must rename their keys: nisse-dev.example.yaml and tests/integration/nisse-test.yaml are updated here. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-01 23:04:37 +00:00
dongarandClaude Opus 5.5	1ad0b2147d	Retire the old build spec, and point every reference at the architecture (M0.1) ... Development Plan M0.1. Every section reference in code, tests, specs/, the example config, the tuning guide, and the CLAUDE.md and README files now cites docs/Architecture.md (P, L, D, O, X, Q) instead of the old spec's A-H, and docs/Architecture-Build-Spec.md is deleted (git keeps it). The plan's old-to-new table goes with it. Mostly mechanical, with a per-site choice where one old section split: A4 became L2 where it meant stop/abandon/the loop and L3 where it meant approvals; A10 became L6 in the context-budget code and L7 elsewhere; B2 became D4 for outbound Gitea and publishing and D5 for review intake; 'A3 step N' became the matching step of L8's boot sequence. Comments describing behaviour the architecture drops (planning always escalating, thresholds tuned at an old build step) now describe the current code and point at the Development Plan. docs/Architecture.md gains one line the rewrite missed: intake outages delay but never fail work (old B2), now in L5, where the poller and Matrix code cite it. Comments and docs only; no behaviour change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE	2026-10-01 00:17:48 +00:00
dongarandClaude Opus 5.5	d49435d4e5	Render one assistant message per turn, and make the request timeout a setting ... Two setup failures found while calibrating the new benchmark tasks, both of which would also fail real milestones. Devstral's chat template (Mistral) rejected any turn in which the model wrote some text before a tool call: the OpenAI renderer made one message per event, so the turn became a text message and a tool-call message in a row, and llama.cpp answered 500. The renderer now folds consecutive user or assistant messages into one, which is the OpenAI format for a turn anyway. Checked against the live server: the captured failing request no longer occurs, over six real runs. LiteLLMClient gave up on any request after a fixed 120 s. A dense 27B model writing a whole module in one turn takes longer, so the executor escalated work that was going well. llm.request_timeout_s now sets it for both clients, default 600. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-27 22:15:30 +00:00
dongarandClaude Opus 5.5	50b46821d7	Add an executor benchmark that keeps its results ... Choosing a local executor model, and tuning its knobs, needs measured success rates across repeated runs, comparable with every model measured before. - Four canned milestones under nisse_dev/bench/tasks: write a function, fix a bug a failing test exposes, thread an option through several files, and find one failure in output far past max_output_chars. Each is an approved plan over a seed project, judged by a hidden check the model never sees; a reference solution proves each is solvable and that its check tells the difference - Runs use the agent's own client wiring, now shared as make_llm_clients, with delegation and ask_expert off: an escalation is a failure of the model under test - Each run is stored as one JSON line in bench/results/, committed, with the model's backing name and declared facts, a note on how it was served, the task's content hash, the agent's commit and tuning, the outcome, turns, misbehaviour counts, context trims, tokens, time and LiteLLM's cost; a fallback answer can never count as a success - The report rebuilds the comparison from every stored run, with Wilson 95% intervals, marking runs against older task versions - scripts/benchmark.py runs and reports; scripts/evaluate_model.sh takes a new model through the context probe, the capability matrix and the benchmark in one command - Clients keep LiteLLM's per-call cost, and BudgetedClient counts the requests it had to trim - The tuning guide gains "Benchmarking models" Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-25 00:35:24 +00:00
dongarandClaude Opus 5.5	b934c46714	Enforce the tool-call contract, asking servers only where it works ... The executor ended a milestone on any reply without a tool call, and nothing held a model to the offered tools. Measured on the 4070: tool_choice "required" is enforced by vLLM (local-fast) and Anthropic, but qwen3-14b (local-coder) under it cannot stop and writes until the output ceiling, a known model bug. So enforcement is asked for per model, and the loop holds the contract itself everywhere. - The catalog reads supports_tool_choice from LiteLLM's model_info; BudgetedClient sends tool_choice only to models declared true, as "required" on the normalized route and {"type": "any"} on the Anthropic passthrough, and only when tools are offered - The executor gains an explicit finish tool, the only way to end a milestone. Finishing with nothing changed is refused once; a prose reply gets one nudge, then escalates; a call to a tool not offered counts toward failure_limit, like a failed command; calls after finish are recorded as not run - Clients record the model that actually answered. ModelFacts.answered recognises a proxy fallback, which once credited Claude's answers to a local model that could not call tools at all - The context probe reports a fallback instead of trusting it; rerun, both local models now refuse over-long prompts, as they should - The capability matrix checks every answer came from the model asked, that models call only offered tools, and that a declared supports_tool_choice is true - Spec A15 and the tuning guide say how enforcement is requested, declared and backed up Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-24 22:54:49 +00:00
dongarandClaude Opus 5.5	494715efa9	Fit every model request to its context window ... Every turn re-sent the whole transcript with nothing checking it against the model's window. On a local model that fails silently: Ollama drops the start of an over-long prompt, which is where the system prompt and the plan are. Measured on the Jetson's llama3.2, cut at 4096 tokens with no error. - The model catalog reads max_input_tokens and max_output_tokens from LiteLLM's model_info, and startup refuses a role on a model with no declared window - BudgetedClient wraps both routes. It fits the canonical events before rendering: older tool results cut to head and tail, then the oldest turns dropped with a note, keeping calls with their results; a request that still cannot fit fails with ContextOverflow and is never sent. The output ceiling is the smaller of config and the model's limit - Tokens are estimated from characters, pessimistically, and the estimate is corrected from reported prompt sizes, only ever upward - The executor takes its scope from the log: the latest plan, which is the approved milestone plan, goes in its system prompt, and it sees only events after it - Tool output keeps its head and, mostly, its tail, so a test runner's failure summary survives truncation - tuning.context.keep_recent_turns and safety_margin are new knobs - scripts/probe_context.py checks a declared window against what the server does; it confirmed llama3.2 cuts at the declared 4096 - The tuning guide covers the budget, declaring a local model's limits, and the probe Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-24 00:26:30 +00:00
dongarandClaude Opus 5.5	df449c7e34	Move configuration to a YAML file, and make the tuning knobs configurable ... The thresholds Part H says to tune on the real inference hardware were hardcoded, so every adjustment would have been a PR. They now live in a `tuning:` section of a YAML config file, which also replaces flat environment variables as the home for structure (#32). - The file (NISSE_CONFIG, default ./nisse-dev.yaml) carries structure and holds no secrets: credentials are ${VAR} references resolved from the environment, and a reference to an unset variable is an error - The environment still wins: every existing NISSE_* variable overrides its key, and the file is optional, so .env and CI work unchanged - Validation stays fail-fast and reports every problem at once; unknown keys are errors, so a misspelt knob fails at startup instead of silently doing nothing - The config file joins the state store and .env in the check that refuses to start when they sit inside the workspace clone - Tuning covers the executor's turn and failure limits, the consult budget, the delegation caps, the command timeout and the tool output limit, which Workspace now takes as a setting; the defaults are the values the agent already ran with - llm.max_output_tokens reaches both LLM clients from the config docs/Tuning-Guide.md explains the knobs for someone learning local model tuning: how weights, quantization and the KV cache spend VRAM, what each knob does and what goes wrong at either extreme, and an order to tune in. nisse-dev.example.yaml documents every key and is checked by a test so it cannot drift. Closes #32 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-23 23:14:07 +00:00
dongarandClaude Opus 5.5	abb7fa0216	Add the operator kill switch, stop (A4) ... `stop` could not have worked by adding the command alone. Planning turns, milestones and review responses ran synchronously on the thread that delivered the message, so the Matrix sync loop read nothing until the work finished. The state store was also opened on the main thread and used from the transport and poller threads, which SQLite refuses with its default same-thread check; the in-memory test transport never showed it. A single worker thread now runs every handler and owns the store: - The worker opens the store, so the same-thread check stays on and fails loudly if anything touches the store from elsewhere - Matrix messages are handed to the worker; pollers hand over and wait, so their cursor never passes an unhandled comment - A message that arrives during long work gets an immediate "busy; say 'stop' to halt" reply and is recorded, not applied later in a state the operator was not answering - Tests keep running inline through InlineWorker, which defers a job submitted from inside another as the real worker would queue it `stop` sets a shared Cancel flag straight from the transport's thread. The executor and delegation loops check it before each model call and between tool calls; the planner checks it before each turn. Calls left unrun get a result saying so, so the transcript still renders. A stopped delegation is discarded like a failed one. The orchestrator moves to AWAIT_INPUT and asks why the work was stopped. The reply is recorded for the model, and a stopped planning turn resumes with it as the answer. `stop` with nothing running says so and changes nothing; at a gate it is no longer taken as rejection feedback. Closes #40 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-23 22:47:29 +00:00
dongarandClaude Opus 5	e38adf40ca	Discover a model's vendor from LiteLLM instead of its name ... Route selection sniffed model names: anything starting "claude-" or "anthropic/" took the vendor-native passthrough, everything else took the normalized route. That works only while an alias names its vendor. A semantic role alias like "remote-architecture" is meant to point at Anthropic today and somewhere else tomorrow, and the name says nothing either way -- so it would have taken the normalized route regardless of what was behind it, silently losing the fidelity A13 wants for delegations. LiteLLM already knows the answer. /model/info reports each alias's backing model, provider, and declared tool support, so the agent asks at startup and routes on the provider it discovers. That keeps vendor knowledge where A5 already put it -- in the proxy -- rather than duplicating it as name patterns here. There is deliberately no name-based fallback: PASSTHROUGH_PREFIXES and has_native_adapter are gone, and select_client now requires a catalog. A second heuristic could only ever disagree with the proxy. An unknown model raises instead of guessing a route. Misconfiguration is therefore loud and early. build_agent validates every role against the catalog before it clones anything, naming all the bad ones at once. Discovery is per key -- /model/info returns only what the presented token may use -- so the same check catches a role pointed at a real model this key cannot reach. An unreachable proxy is reported separately, since an outage and a typo call for different responses from the operator. Two things now get recorded rather than inferred: the resolved model is pinned alongside the role mapping at task start, for the reason A11 already pins roles, and DelegationStarted carries backing_model so A16 eval history is not averaged across whatever an alias happened to point at. A model LiteLLM declares tool-less warns at startup rather than refusing; A15 expects weak local models to be accommodated, and they are observed driving tool loops anyway. Verified live: all four aliases the production key can see route correctly through discovery, and each chosen route works. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-23 01:47:58 +00:00
dongarandClaude Opus 5	ed2f5dfc7f	Choose the LLM route by model, and add a capability matrix ... The arrival of non-Anthropic models made a latent bug observable: the delegate's client was hardcoded to the Anthropic passthrough, so pointing NISSE_ROLE_DELEGATE at gpt-6-astra or a local model would have sent Anthropic-shaped /v1/messages bodies to something that does not speak them. A5 promises the frontier tier is swappable "as a configuration change, not a refactor", and it was not. select_client now picks the route from the model: the vendor-native adapter where one exists, LiteLLM's normalization otherwise — which is exactly what A13 prescribes for a vendor whose renderer adapter has not been written, pending validation against the A16 eval set. Also adds the capability matrix Part H asks for. It says the thresholds that depend on local capability — A12's failure count, A14's consult budget, A15's manifest size — must be tuned "on the real hardware", and that needs measurement rather than reputation. The harness probes each model for the things that actually break: whether it calls a tool at all, whether the plan it submits has substance (llama3.2 emitted "{"), whether it can consume its own tool result, and whether prompt-level JSON holds. Opt in with NISSE_CAPABILITY_MODELS; it spends real tokens across several models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-22 23:20:38 +00:00
dongarandClaude Opus 5	2fb25b8195	Add the composition root and a runnable entry point ... Nothing had ever assembled the agent. main.py was still the hello-world placeholder from the scaffold, and no code outside the test suite constructed an Orchestrator — which is how two fully built, fully tested components came to be reachable from nothing: the delegation harness, and the Gitea comment poller. Unit tests that construct a component directly cannot catch that; only assembly can. - config.py validates every variable before anything is built and names all of them at once, rather than failing at first use twenty minutes into a milestone on the call that mattered. Whitespace-only counts as missing, since a token of spaces passes a truthiness check and fails at the first request instead - agent.py builds the whole graph and owns startup and shutdown order: reconcile before accepting commands, and on stop, stop watching, then stop talking, then close the store last - The Gitea account is read via whoami rather than configured: it names the identity every permission is evaluated against, and proves the token works before anything depends on it - Poller lifecycle is now a state-machine concern: watch the plan issue while a gate stands, the PR while it is open, and stop on leaving. The orchestrator owns when to watch; an injected factory owns how to build one - The executor reports each turn, because it runs synchronously inside the transport callback: without it the agent is silent for a whole milestone and a working run looks exactly like a hung one - main.py loads .env, builds, and shuts down cleanly on SIGINT/SIGTERM The entry-point tests changed shape: nisse-dev no longer greets, it refuses to start unconfigured and says everything that is missing. Both tests now run with a deliberately empty environment and a .env path that does not exist, because the working directory's .env holds live credentials and a test must never be one cwd away from starting a real agent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-08 22:26:26 +00:00