M3.2: the action classifier
What the fast path calls unknown is judged by an independent model call on the item's classifier role (L4), before it runs. - nisse_dev.loop.classifier: it sees the action and the agreed scope (project, repository, issue number, the restatement the operator said go to), built in code; never the transcript, the issue's text, tool output or the driver's reasoning. Its verdict is a tool call: safe runs; in_scope runs and is reported in the thread; needs_discussion asks, as a dangerous action does. An error, no verdict or an unknown one fails closed. Verdicts are cached per item, agreed revision and action. A helper is refused what needs discussion. - action_verdict gains source, model and the classifier's exact input; details shows the source. An allow after the classifier asked is an override. scripts/classifier-evals.py exports the judgements from a store, read-only; tests/integration/classifier_cases.yaml is the seed set, run per model with --capability-models. - The fast path's delete rule: deleting on the build VM is contained; only what breaks the host (system and home directories) stays dangerous, since in tests the build VM is the test machine. - The classifier role is the local model in the example config. - The property walk gains a classifier answering at random, junk and failures included; dangerous_only_when_allowed now covers its verdicts (checked against a mutant that runs them all). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
1 parent
5044af371b
commit
089af0c391
15 files changed
+1023
-71
No files matched your search
@@ -739,7 +739,10 @@ Installing on any other host is dangerous.
|
||||
| Dangerous | Merging to the default branch (unless the trust dial allows), deleting remote branches or others' work, changing repository settings, anything outside the item's repository or build VM |
|
||||
|
||||
Changing branches by hand is neither: it is refused, with a pointer to the
|
||||
milestone tools, since the harness owns branches (D4).
|
||||
milestone tools, since the harness owns branches (D4). Deleting on the build VM
|
||||
is contained, as changing it is; only what would break the host (system and
|
||||
home directories, `dd`, `mkfs`, `reboot`) is dangerous, since in tests the
|
||||
"build VM" is the test machine itself.
|
||||
|
||||
### D7. Deployment and isolation
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ Only `nisse-dev` has real code; `nisse-core` and `nisse-ops` are placeholders.
|
||||
| **L1** Work model | mostly done | Work items with an active pointer (M1.1); issue intake (M2.1); queue, parking and blocked items (M2.2) on the new loop; no grace period yet |
|
||||
| **L2** Agent loop | mostly done | The loop (Phase 2), the only one since the switch-over (M2.8): driver runs, waits, `stop`, `abandon`, question runs |
|
||||
| **L3** Authority | partial | Go-aheads bound to the restatement's revision, on the issue and its conversation (M2.1); action classes and the fast path, with `allow` bound to one dangerous action (M3.1); no trust dial yet (M3.3) |
|
||||
| **L4** Classifier | none | The fast path's verdicts are logged for its evaluation set (M3.1); unknown actions run until it exists (M3.2) |
|
||||
| **L4** Classifier | mostly done | On the classifier role (the local model), for what the fast path doesn't know; fails closed; overrides export as an evaluation set (M3.2). Promoting its verdicts into rules is by hand |
|
||||
| **L5** Conversation | partial | Matrix transport, mentions, lenient commands; on the new loop, a thread per item, named and ambiguous commands, plain-language go-aheads (M2.3); little explanation |
|
||||
| **L6** Context | partial | Context fitting, per-role views and orchestrator facts (M1.2); no scope or agent memory |
|
||||
| **L7** Canonical log | partial | Work item and actor on every event, `fact` (M1.1, M1.2); `action_verdict` (M3.1); no `knowledge_retrieved` |
|
||||
@@ -331,8 +331,29 @@ Harmless, contained and dangerous actions per agent (P3, D6), static fast-path r
|
||||
- Deferred: the classifier and in-scope judgement (M3.2), the trust dial
|
||||
(M3.3), egress enforcement (X2).
|
||||
|
||||
**M3.2 The classifier.** *Needs discussion.* L4, its model, fail-closed verdicts,
|
||||
overrides as an evaluation set.
|
||||
**M3.2 The classifier.** *Needs discussion.* **Done** (this PR). L4, its model,
|
||||
fail-closed verdicts, overrides as an evaluation set.
|
||||
- `nisse_dev.loop.classifier`: one model call per action the fast path calls
|
||||
unknown, on the item's pinned `classifier` role. It sees the action and the
|
||||
agreed scope (project, repository, issue number, the restatement the operator
|
||||
said go to), built in code: never the transcript, the issue's text, tool
|
||||
output or the driver's reasoning. Its verdict is a tool call: `safe` runs,
|
||||
`in_scope` runs and is reported in the thread, `needs_discussion` asks as a
|
||||
dangerous action does. An error, no verdict or an unknown one fails closed.
|
||||
- Decided: no different-model requirement (separate context is what L4's
|
||||
independence needs; the verdicts record the model, so correlated misses can
|
||||
be measured); the local model to start with; verdicts cached per item,
|
||||
agreed revision and action.
|
||||
- `action_verdict` gains `source` (fast path or classifier), `model` and the
|
||||
classifier's exact `input`; `details` shows the source. An `allow` after the
|
||||
classifier asked is an override. `scripts/classifier-evals.py` exports the
|
||||
judgements from a store (read-only); `tests/integration/classifier_cases.yaml`
|
||||
is the seed set, run per model with `--capability-models`
|
||||
(`qwen3.8-27b-nothink` and `claude-haiku-4-5`: 9 of 9 each).
|
||||
- The fast path's delete rule: on the build VM, deleting is contained; only
|
||||
what breaks the host stays dangerous (decided with the operator).
|
||||
- Deferred: promoting verdicts the classifier keeps agreeing on into rules (by
|
||||
hand, from exported cases); the trust dial (M3.3).
|
||||
|
||||
**M3.3 The trust dial.** *Needs discussion.* Per project (D8); agent merging behind
|
||||
protection it can't satisfy, verified by a test (D4, D5, #65).
|
||||
|
||||
Reference in new issue
Block a user