dev/nisse
2
1 Fork 0
Code Issues 23 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Test levels: smoke e2e on PRs, full set on e2e:full and before a release #117

Merged
dev-agent merged 3 commits from test-tiers into main 2026-10-07 22:55:15 +00:00
Conversation 2 Commits 3 Files Changed 9
+152 -29
3 Commits
Author SHA1 Message Date
dongarandClaude Opus 5.5 28092194fe E2E on qwen3.8-27b without thinking
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m14s
Details
CI / e2e (pull_request) Successful in 3m6s
Details 
The e2e roles move to qwen3.8-27b-nothink, a LiteLLM alias for the same
model with chat_template_kwargs enable_thinking false. With thinking, a
growing share of answers ran to 2-8k tokens (two hit the 8192 cap), which
took one test from 3.5 to 20 minutes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
 2026-10-07 22:49:13 +00:00
dongarandClaude Opus 5.5 ff5a7a644f CI: install sshd from a fast mirror, or not at all
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 2m10s
Details
CI / e2e (pull_request) Successful in 9m32s
Details 
archive.ubuntu.com has been crawling (a few KB/s), taking the sshd install
in every job from 8 seconds to 3.5-9.5 minutes (another agent measured it;
the runner and inference-4090 were healthy). scripts/ci-sshd.sh skips the
install when the image has sshd, and otherwise points apt at
mirrors.edge.kernel.org first (both source layouts, 22.04 and 24.04).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
 2026-10-07 22:29:41 +00:00
dongarandClaude Opus 5.5 e35aea5045 Test levels: smoke e2e on PRs, the full set on e2e:full and before a release 
- Every PR: lint, unit tests across the runner's cores (pytest-xdist), 16
  property walks from a seed that moves with the run number, integration
  tests, and the end-to-end smoke tests (the slice, a conversation filed
  as work, a question; `smoke` marker).
- A PR labelled e2e:full, a push to main, or a manual run: all 48 walks and
  the whole end-to-end suite. Adding e2e:full to a PR runs it; any other
  label runs nothing.
- A published release runs the full set before its image is built.
- The walk's seeds come from NISSE_WALK_SEEDS / NISSE_WALK_FIRST (48 from 0
  by default).

Measured here: the unit tier on 2 workers took 3m49s against 6-8 minutes
serially.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
 2026-10-07 22:29:41 +00:00