Use whatmodel.sh to suggest model updates #150

New Issue

2026-10-15T11:08:16Z

dev-agent commented

2026-10-15 11:08:16 +00:00

The operator wants to start using the whatmodel.sh leaderboard as a source of suggestions for model updates:

AI model leaderboard: the best AI models you can run · whatmodel.sh — https://share.google/vTQVizL9PVnQ3zHky

Each suggestion would name a candidate for one of the agents' roles (driver, helper, expert, classifier, reviewer). It would then go through a benchmark before any role changes, as qwen3.8-27b did in PR #55.

To settle:

Who checks the leaderboard, and how often. An agent might do it later, perhaps nisse-ops as a duty (O5).
What turns a candidate into a benchmark run: for example, it fits the 4090, LiteLLM can serve it, and it beats the current pick for a role.
Where suggestions land: an issue per candidate, or a comment here.

First candidate, already noted: Qwen3.8-Flash-Next, to be benchmarked against qwen3.8-27b.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

The operator wants to start using the whatmodel.sh leaderboard as a source of suggestions for model updates: - AI model leaderboard: the best AI models you can run · whatmodel.sh — https://share.google/vTQVizL9PVnQ3zHky Each suggestion would name a candidate for one of the agents' roles (driver, helper, expert, classifier, reviewer). It would then go through a benchmark before any role changes, as `qwen3.8-27b` did in PR #55. To settle: - Who checks the leaderboard, and how often. An agent might do it later, perhaps nisse-ops as a duty (O5). - What turns a candidate into a benchmark run: for example, it fits the 4090, LiteLLM can serve it, and it beats the current pick for a role. - Where suggestions land: an issue per candidate, or a comment here. First candidate, already noted: **Qwen3.8-Flash-Next**, to be benchmarked against `qwen3.8-27b`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/nisse#150