dongarandClaude Opus 5.5 e35aea5045 Test levels: smoke e2e on PRs, the full set on e2e:full and before a release ...

- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16
  property walks from a seed that moves with the run number, integration
  tests, and the end-to-end smoke tests (the slice, a conversation filed
  as work, a question; `smoke` marker).
- A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and
  the whole end-to-end suite. Adding e2e:full to a PR runs it; any other
  label runs nothing.
- A published release runs the full set before its image is built.
- The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0
  by default).

Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes
serially.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE

2026-10-07 22:29:41 +00:00

..

seed

A repository per test run instead of the shared sandbox

2026-10-06 05:10:27 +00:00

conftest.py

Pass the capability matrix's models as a pytest option

2026-09-23 23:32:44 +00:00

e2e_support.py

New projects from conversation (M2.5, PR B)

2026-10-07 15:39:00 +00:00

nisse-test.yaml

New projects from conversation (M2.5, PR B)

2026-10-07 15:39:00 +00:00

test_benchmark_live.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_crash_durability.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_delegation_live.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_end_to_end_loop.py

Test levels: smoke e2e on PRs, the full set on e2e:full and before a release

2026-10-07 22:29:41 +00:00

test_end_to_end.py

End-to-end tests on a local model, and in CI on every PR

2026-10-07 15:14:43 +00:00

test_entry_points.py

Entry-point test: a config with no projects is named in the startup errors

2026-10-06 02:36:06 +00:00

test_expert_live.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_fresh_repo_live.py

A repository per test run instead of the shared sandbox

2026-10-06 05:10:27 +00:00

test_gate_restart.py

Retire the old build spec, and point every reference at the architecture (M0.1)

2026-10-01 00:17:48 +00:00

test_gitea_live.py

A repository per test run instead of the shared sandbox

2026-10-06 05:10:27 +00:00

test_llm_live.py

Don't assert the exact word in the LiteLLM smoke test

2026-09-29 17:22:24 +00:00

test_matrix_live.py

Matrix threads in the transport (M2.1, PR A)

2026-10-02 23:40:24 +00:00

test_milestone_live.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_model_capability.py

Retire the old build spec, and point every reference at the architecture (M0.1)

2026-10-01 00:17:48 +00:00

test_passthrough_live.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_planning_live.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00