Commit Graph
2 Commits
Author SHA1 Message Date
dongarandClaude Opus 5.5 97863c1939 Bind approvals and feedback to the revision they were about
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m16s
Fixes counterexamples 1-4 from specs/orchestrator.qnt: a repeated
'approve', 'approve rev-1' meant for the architecture, approving on both
channels, and 'approve' plus an afterthought all landed on the next
plan, one the operator had not seen.

- Revisions are numbered across the work item (A4 already required
  monotonic tags; the code numbered per gate), so rev-N names one plan.
- Each reply is stamped on arrival with the revision that was up: in
  Matrix the one whose approval request had been sent (a plain value
  the transport thread reads, restored on boot), in Gitea the latest
  plan comment older than the reply (board.posts).
- Gate.evaluate applies a reply that names no revision only if its stamp
  is the current revision. Otherwise an approval is not applied and
  feedback rejects nothing (it is still in the transcript), and the
  agent says which revision is up. 'approve rev-N' binds by its number.
- Spec A4 states both rules. The model applies them; scenarios 1-4 now
  expect the properties to hold, and both held over 200,000 random
  traces and Apalache's exhaustive check to depth 8.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-09-29 19:13:20 +00:00
dongarandClaude Opus 5.5 d79e399933 Model the orchestrator in Quint, and reproduce its counterexamples
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m14s
specs/orchestrator.qnt models control.py: states, gate, pending
questions, comment cursors, publish intents and the worker queue, with
operator messages, Gitea comments, work outcomes (exceptions included)
and a crash at any point interleaving freely. Eight properties.

Seven counterexamples, each written out as a trace in
specs/orchestrator_scenarios.qnt and reproduced against the real code in
nisse-dev/tests/test_quint_counterexamples.py (xfail, strict, until
fixed):

1-4 an approval or feedback lands on a plan the operator had not seen:
    a double 'approve', 'approve rev-1' meant for the architecture,
    approving on both channels, and 'approve' plus an afterthought;
5   a crash after the PR opens but before PR_OPEN publishes again and
    cannot get back to PR_OPEN;
6   a crash while answering a review comment loses it silently;
7   'project complete' leaves the plan issue open.

no_stranding, await_input_has_a_question and execution_needs_approval
held over 200,000 random traces of 40 steps, and under Apalache's
exhaustive check to depth 8, which does find counterexample 1 at that
depth. No behaviour changes here.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
2026-09-29 18:51:59 +00:00