Test levels: smoke e2e on PRs, the full set on e2e:full and before a release
- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16 property walks from a seed that moves with the run number, integration tests, and the end-to-end smoke tests (the slice, a conversation filed as work, a question; `smoke` marker). - A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and the whole end-to-end suite. Adding e2e:full to a PR runs it; any other label runs nothing. - A published release runs the full set before its image is built. - The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0 by default). Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes serially. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
1 parent
07a2eb9d10
commit
e35aea5045
7 files changed
+124
-15
No files matched your search
@@ -4,6 +4,7 @@ members = ["nisse-core", "nisse-dev", "nisse-ops"]
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"pytest>=8",
|
||||
"pytest-xdist>=3",
|
||||
"ruff>=0.6",
|
||||
]
|
||||
|
||||
@@ -19,6 +20,7 @@ addopts = "--strict-markers --ignore=tests/integration/seed"
|
||||
markers = [
|
||||
"integration: end-to-end tests that exercise installed entry points or external services",
|
||||
"e2e: the whole agent against real services, run after each milestone (scripts/e2e.sh)",
|
||||
"smoke: the end-to-end tests every PR runs; the rest on e2e:full and before a release",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
|
||||
Reference in new issue
Block a user