dongarandClaude Opus 5.5
4b6c48cc96
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m12s
Five runs per task at 320 W with max_turns 40, after the fixes in #50-#52, for qwen3.8-27b, qwen3.6-35b-a3b, gemma-4-26b-a4b, devstral-small-2-24b and mimo-v2.6-9b; the harder four also for claude-haiku-4-5 as the ceiling, and two calibration runs each for local-fast as the floor. Runs failed by the proxy's rate limiter were set aside, not stored. Earlier results move to bench/results/archive/, which the report does not read: they were made at other power caps, at max_turns 24, or on the RTX 4070, and are kept as the record. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
928 B
928 B
Archived benchmark results
Runs set aside on 2026-09-27. They are kept as the record of how the local
model choice was made, but the report no longer reads them: benchmark.py report only looks at bench/results/*.jsonl, not this folder.
- 2026-09-25,
local-coderandlocal-fast: qwen3-14b and qwen3-8b-awq on the original RTX 4070 box. Both models were later dropped. - 2026-09-27, seven newer models and
local-faston the RTX 4090 (PR #49): every newer model passed all four original tasks 20/20, which is why four harder tasks were added. Pass rates are still valid; timings are not comparable with later runs, which were made under a 320 W power cap rather than 450 W or 350 W, withmax_turns40 rather than 24.
Later results were made after the fixes in PRs #50–#52: rate-limit waits recorded, one assistant message per turn, a 600 s request timeout, and tuning kept apart in the report.