Test levels: smoke e2e on PRs, the full set on e2e:full and before a release
- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16 property walks from a seed that moves with the run number, integration tests, and the end-to-end smoke tests (the slice, a conversation filed as work, a question; `smoke` marker). - A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and the whole end-to-end suite. Adding e2e:full to a PR runs it; any other label runs nothing. - A published release runs the full set before its image is built. - The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0 by default). Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes serially. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
This commit is contained in:
1 parent
07a2eb9d10
commit
e35aea5045
7 files changed
+124
-15
No files matched your search
@@ -3,7 +3,8 @@ name: Release
|
||||
# registry (Development Plan M1.5, X4): gitea.dongar.net/dev/nisse-dev
|
||||
# tagged with the version (the release tag without its v) and latest. Cut a
|
||||
# release in Gitea (Releases → New release, tag vX.Y.Z); a tag pushed for any
|
||||
# other reason builds nothing.
|
||||
# other reason builds nothing. The full test set runs first: no image without
|
||||
# it passing.
|
||||
#
|
||||
# The registry needs a personal access token with write:package
|
||||
# (REGISTRY_TOKEN, an organization secret of `dev`, belonging to the
|
||||
@@ -23,7 +24,50 @@ env:
|
||||
IMAGE: gitea.dongar.net/dev/nisse-dev
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
# Everything, before an image exists: lint, unit tests with all 48
|
||||
# property walks, integration tests and the whole end-to-end suite.
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 150
|
||||
env:
|
||||
NISSE_LLM_TOKEN: ${{ secrets.NISSE_LLM_TOKEN }}
|
||||
NISSE_MATRIX_TOKEN: ${{ secrets.NISSE_MATRIX_TOKEN }}
|
||||
NISSE_GITEA_TOKEN: ${{ secrets.NISSE_GITEA_TOKEN }}
|
||||
NISSE_OPERATOR_TOKEN: ${{ secrets.NISSE_OPERATOR_TOKEN }}
|
||||
NISSE_E2E_RUNS: /tmp/nisse-e2e-runs
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Sync workspace
|
||||
run: uv sync --all-packages
|
||||
|
||||
- name: Install sshd
|
||||
run: |
|
||||
SUDO=$(command -v sudo || true)
|
||||
$SUDO apt-get update -qq
|
||||
$SUDO apt-get install -y -qq openssh-server openssh-client > /dev/null
|
||||
$SUDO mkdir -p /run/sshd
|
||||
|
||||
- name: Lint
|
||||
run: uv run ruff check . && uv run ruff format --check .
|
||||
|
||||
- name: Unit tests
|
||||
run: uv run pytest -n auto -m "not integration and not e2e"
|
||||
|
||||
- name: Integration tests
|
||||
run: uv run pytest -m "integration and not e2e" -rs
|
||||
|
||||
- name: End-to-end tests
|
||||
run: scripts/e2e.sh --local -v
|
||||
|
||||
build-and-push:
|
||||
needs: tests
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
Reference in new issue
Block a user