Use qwen3.8-27b as the example executor #56

2026-09-29T14:18:50Z

dev-agent commented

2026-09-29 14:18:50 +00:00

Points roles.executor in nisse-dev.example.yaml at qwen3.8-27b, and asserts it in the example-file test.

qwen3.8-27b led the executor benchmark (#55): 37/40 over eight tasks, 17/20 on the harder four, ahead of claude-haiku-4-5 (15/20). The runner-up MoE models were faster but weaker (32/40, 30/40). local-fast, the previous example, scored 0/8 on the harder four. Its cost is time on hard tasks (about 5 to 8.5 minutes a run), which suits a milestone-driven executor.

Selection is on the dev executor alone for now, to get one agent working end to end first. The spec's open item on sharing the local host (Part H) is unchanged.

🤖 Generated with Claude Code

Points `roles.executor` in `nisse-dev.example.yaml` at `qwen3.8-27b`, and asserts it in the example-file test. qwen3.8-27b led the executor benchmark (#55): 37/40 over eight tasks, 17/20 on the harder four, ahead of claude-haiku-4-5 (15/20). The runner-up MoE models were faster but weaker (32/40, 30/40). `local-fast`, the previous example, scored 0/8 on the harder four. Its cost is time on hard tasks (about 5 to 8.5 minutes a run), which suits a milestone-driven executor. Selection is on the dev executor alone for now, to get one agent working end to end first. The spec's open item on sharing the local host (Part H) is unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dev-agent added 1 commit 2026-09-29 14:18:50 +00:00

Use qwen3.8-27b as the example executor

CI / checks (push) Skipped

CI / checks (pull_request) Successful in 1m22s

Details

fe73112aa2

It led the executor benchmark (PR #55): 37/40 over eight tasks and 17/20
on the harder four, ahead of claude-haiku-4-5's 15/20. local-fast, the
previous example, scored 0/8 on the harder four. Model selection is on
the dev executor alone for now; the shared-host question (spec Part H)
stays open.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

dongar approved these changes

2026-09-29 14:20:14 +00:00

dongar merged commit e5ed32bc2b into main

2026-09-29 14:20:25 +00:00

dongar deleted branch executor-qwen3.8-27b

2026-09-29 14:20:25 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/nisse#56