dongarandClaude Opus 5.5
6ae2375194
nisse-dev now runs only the conversational loop. The gated state machine it was built alongside is gone: - Removed control.py, planner.py, executor.py, approvals.py, gates.py (command_form moves to loop/reader.py), publish.py and poller.py, the old assembly in agent.py (its shared helpers move to loop/build.py), the store's state-machine view (get_state, transition), their tests, the old end-to-end and live planning/milestone/gate tests, and specs/orchestrator.qnt with its scenarios. - Roles are driver, helper, expert and classifier; driver is required. loop:, roles.planner, roles.executor and tuning.executor are refused with what to use instead; tuning.driver takes the turn cap and failure limit. - The benchmark and the capability matrix measure the driver: the loop's own system prompt and tools, the task's plan as the agreed restatement. Task plans end with finish_milestone, so earlier runs show as (old); stored runs' executor tuning reads as the driver's. - A delegation on the loop records what its helper's alias resolved to (L9), as the old path's did. - The dump shows a work item's revision and what it waits for. - Behaviour the old tests covered and the loop shares stays covered: driver runs that fail on a model call, stop mid-turn, reset the failure count; go-aheads ignoring case and punctuation; a request too big to fit is never sent; the assembly tests, now on build_loop_agent. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE
35 lines
1.1 KiB
Python
35 lines
1.1 KiB
Python
"""One live benchmark run: the simplest task, through the agent's own
|
|
client wiring, with the test config's driver."""
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from nisse_dev.bench.runner import run_task
|
|
from nisse_dev.bench.tasks import load_tasks
|
|
from nisse_dev.catalog import ModelCatalog
|
|
from nisse_dev.config import AgentConfig
|
|
from nisse_dev.execution import SSHHost
|
|
from nisse_dev.loop.build import make_llm_clients
|
|
|
|
pytestmark = pytest.mark.integration
|
|
|
|
|
|
def test_the_simplest_task_passes_live(
|
|
live_config: AgentConfig, tmp_path: Path, ssh_host: SSHHost
|
|
) -> None:
|
|
catalog = ModelCatalog.discover(live_config.llm_base_url, live_config.llm_token)
|
|
normalized, _ = make_llm_clients(live_config, catalog)
|
|
facts = catalog.facts(live_config.roles.driver)
|
|
[task] = load_tasks(["write-function"])
|
|
|
|
record = run_task(
|
|
task, facts, normalized, live_config.tuning, host=ssh_host, workdir=tmp_path
|
|
)
|
|
|
|
assert record.fallbacks == []
|
|
assert record.success, (
|
|
f"{record.outcome}: {record.outcome_note}\n{record.check_output}"
|
|
)
|
|
assert record.cost >= 0
|