Commit Graph
2 Commits
Author SHA1 Message Date
dongarandClaude Opus 5.5 4b6c48cc96 Benchmark the finalists on all eight tasks, and archive older results
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m12s
Five runs per task at 320 W with max_turns 40, after the fixes in
#50-#52, for qwen3.8-27b, qwen3.6-35b-a3b, gemma-4-26b-a4b,
devstral-small-2-24b and mimo-v2.6-9b; the harder four also for
claude-haiku-4-5 as the ceiling, and two calibration runs each for
local-fast as the floor. Runs failed by the proxy's rate limiter were
set aside, not stored.

Earlier results move to bench/results/archive/, which the report does
not read: they were made at other power caps, at max_turns 24, or on
the RTX 4070, and are kept as the record.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 03:14:40 +00:00
dongarandClaude Opus 5.5 82f171c2f5 Benchmark seven new RTX 4090 models, and local-fast on the 4090
CI / checks (push) Skipped
CI / checks (pull_request) Successful in 1m16s
Five runs per task for qwen3.8-27b, qwen3.6-35b-a3b, gemma-4-31b,
gemma-4-26b-a4b, gemma-4-12b, mimo-v2.6-9b and granite-4.2-8b, all
served by llama.cpp b11206 through llama-swap. Every one passed 20 of
20. local-fast (vLLM 0.30.0) passed 3 of 20.

Runs were made under a 350 W power cap, recorded in each note, after
the uncapped card throttled for heat. granite-4.2-8b also has one
uncapped set, kept apart by its note.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 18:43:55 +00:00