Architecture L9's role names, for the two roles with a direct
counterpart: the delegation role is now 'helper' and the ask_expert role
'expert', in RoleMap, configuration (roles.helper, roles.expert),
recorded events and tests. 'planner' and 'executor' stay distinct for
now; they become 'driver' when the new loop merges them (Development
Plan M2.3).
Configurations must rename their keys: nisse-dev.example.yaml and
tests/integration/nisse-test.yaml are updated here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
Development Plan M0.1. Every section reference in code, tests, specs/,
the example config, the tuning guide, and the CLAUDE.md and README files
now cites docs/Architecture.md (P, L, D, O, X, Q) instead of the old
spec's A-H, and docs/Architecture-Build-Spec.md is deleted (git keeps
it). The plan's old-to-new table goes with it.
Mostly mechanical, with a per-site choice where one old section split:
A4 became L2 where it meant stop/abandon/the loop and L3 where it meant
approvals; A10 became L6 in the context-budget code and L7 elsewhere; B2
became D4 for outbound Gitea and publishing and D5 for review intake;
'A3 step N' became the matching step of L8's boot sequence. Comments
describing behaviour the architecture drops (planning always escalating,
thresholds tuned at an old build step) now describe the current code and
point at the Development Plan.
docs/Architecture.md gains one line the rewrite missed: intake outages
delay but never fail work (old B2), now in L5, where the poller and
Matrix code cite it.
Comments and docs only; no behaviour change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
It led the executor benchmark (PR #55): 37/40 over eight tasks and 17/20
on the harder four, ahead of claude-haiku-4-5's 15/20. local-fast, the
previous example, scored 0/8 on the harder four. Model selection is on
the dev executor alone for now; the shared-host question (spec Part H)
stays open.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The report grouped runs by model, serving note and task version, so
runs made at different tuning were silently averaged together. Tuning
is now part of the key, and a tuning column shows how a row's settings
differ from today's defaults; it appears only when some row needs it.
tuning.executor.max_turns goes from 24 to 40. On the harder benchmark
tasks Haiku, the calibration ceiling, hit 24 on both refactor-rename
runs and both long-milestone runs, one of which had finished the work.
The cap guards against a model trying forever; it isn't a budget.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two setup failures found while calibrating the new benchmark tasks,
both of which would also fail real milestones.
Devstral's chat template (Mistral) rejected any turn in which the model
wrote some text before a tool call: the OpenAI renderer made one message
per event, so the turn became a text message and a tool-call message in
a row, and llama.cpp answered 500. The renderer now folds consecutive
user or assistant messages into one, which is the OpenAI format for a
turn anyway. Checked against the live server: the captured failing
request no longer occurs, over six real runs.
LiteLLMClient gave up on any request after a fixed 120 s. A dense 27B
model writing a whole module in one turn takes longer, so the executor
escalated work that was going well. llm.request_timeout_s now sets it
for both clients, default 600.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Planner and delegate use claude-opus-5-5 for the high-end judgement
those roles need. The consultant answers single questions with no tools,
so it uses claude-haiku-4-5. Sonnet has no role.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every turn re-sent the whole transcript with nothing checking it against
the model's window. On a local model that fails silently: Ollama drops
the start of an over-long prompt, which is where the system prompt and
the plan are. Measured on the Jetson's llama3.2, cut at 4096 tokens with
no error.
- The model catalog reads max_input_tokens and max_output_tokens from
LiteLLM's model_info, and startup refuses a role on a model with no
declared window
- BudgetedClient wraps both routes. It fits the canonical events before
rendering: older tool results cut to head and tail, then the oldest
turns dropped with a note, keeping calls with their results; a request
that still cannot fit fails with ContextOverflow and is never sent.
The output ceiling is the smaller of config and the model's limit
- Tokens are estimated from characters, pessimistically, and the
estimate is corrected from reported prompt sizes, only ever upward
- The executor takes its scope from the log: the latest plan, which is
the approved milestone plan, goes in its system prompt, and it sees
only events after it
- Tool output keeps its head and, mostly, its tail, so a test runner's
failure summary survives truncation
- tuning.context.keep_recent_turns and safety_margin are new knobs
- scripts/probe_context.py checks a declared window against what the
server does; it confirmed llama3.2 cuts at the declared 4096
- The tuning guide covers the budget, declaring a local model's limits,
and the probe
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With no production instance yet, there is no reason to carry two ways
of configuring the agent. Every environment variable that duplicated a
config key is gone; the environment now supplies only the secrets the
file references by ${…}, plus NISSE_CONFIG and NISSE_ENV to find the
files.
- AgentConfig loads from a file (load, from_file) or a parsed tree
(from_tree, for tests); a missing file is an error naming the example
- The from_env constructors on LiteLLMClient, AnthropicPassthroughClient,
GiteaClient, MatrixConfig and RoleMap are removed, with the env
reading for max output tokens; clients take their settings from config
- The live integration tests share a committed agent config,
tests/integration/nisse-test.yaml, aimed at test resources, and one
conftest that loads it and builds clients. NISSE_LIVE_PLANNER_MODEL
becomes its planner role; NISSE_CAPABILITY_MODELS stays as a per-run
switch
- A missing secret now fails the live tests instead of skipping them
- CI passes only the three tokens
- .env.example is secrets only; spec A5 says config comes from the file
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The thresholds Part H says to tune on the real inference hardware were
hardcoded, so every adjustment would have been a PR. They now live in a
`tuning:` section of a YAML config file, which also replaces flat
environment variables as the home for structure (#32).
- The file (NISSE_CONFIG, default ./nisse-dev.yaml) carries structure
and holds no secrets: credentials are ${VAR} references resolved from
the environment, and a reference to an unset variable is an error
- The environment still wins: every existing NISSE_* variable overrides
its key, and the file is optional, so .env and CI work unchanged
- Validation stays fail-fast and reports every problem at once; unknown
keys are errors, so a misspelt knob fails at startup instead of
silently doing nothing
- The config file joins the state store and .env in the check that
refuses to start when they sit inside the workspace clone
- Tuning covers the executor's turn and failure limits, the consult
budget, the delegation caps, the command timeout and the tool output
limit, which Workspace now takes as a setting; the defaults are the
values the agent already ran with
- llm.max_output_tokens reaches both LLM clients from the config
docs/Tuning-Guide.md explains the knobs for someone learning local
model tuning: how weights, quantization and the KV cache spend VRAM,
what each knob does and what goes wrong at either extreme, and an order
to tune in. nisse-dev.example.yaml documents every key and is checked
by a test so it cannot drift.
Closes#32
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
dongarandClaude Opus 5.5