Test levels: smoke e2e on PRs, the full set on e2e:full and before a release

- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16
  property walks from a seed that moves with the run number, integration
  tests, and the end-to-end smoke tests (the slice, a conversation filed
  as work, a question; `smoke` marker).
- A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and
  the whole end-to-end suite. Adding e2e:full to a PR runs it; any other
  label runs nothing.
- A published release runs the full set before its image is built.
- The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0
  by default).

Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes
serially.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
dongarandClaude Opus 5.5 committed 2026-10-07 22:29:41 +00:00
1 parent 07a2eb9d10
commit e35aea5045
7 files changed
+124 -15

No files matched your search

+9 -2
View File
@@ -8,6 +8,7 @@ checks every property after every step. A failure prints
the seed and the steps, so it replays exactly.
"""
import os
import random
import re
import subprocess
@@ -46,8 +47,14 @@ from nisse_dev.tools import Workspace
from nisse_dev.transport import InboundMessage, InMemoryTransport
from nisse_dev.worker import InlineWorker
SEEDS = range(32)
TWO_PROJECT_SEEDS = range(100, 116)
# How many walks, and from which seed: all 48 by default; CI runs fewer on a
# PR, from a seed that moves with the run number, so PRs still cover new
# ground (NISSE_WALK_SEEDS, NISSE_WALK_FIRST). Two thirds walk one project,
# a third two.
_WALKS = int(os.environ.get("NISSE_WALK_SEEDS") or 48)
_FIRST = int(os.environ.get("NISSE_WALK_FIRST") or 0)
SEEDS = range(_FIRST, _FIRST + _WALKS * 2 // 3)
TWO_PROJECT_SEEDS = range(100 + _FIRST, 100 + _FIRST + _WALKS - _WALKS * 2 // 3)
STEPS = 30