Compaction: the model's notes replace the middle of a long run (#158)
Before each driver call, over compact_at (0.6) of the prompt budget, the same model is asked for notes over the very request it was about to get, plus one message, so the server's cache covers all but the notes. The notes, with code's list of the calls they replace, become a compaction event that renders in place of its range, keeping word for word what started the work and the last few turns. Rare, large steps keep the prompt's start unchanged between them. A later compaction folds in the earlier; one from an earlier run's view is hidden. Failed notes are logged and fitting takes over. Fitting now applies compactions before it measures, records what it cuts (Trim) and logs each cut as context.trimmed. Wired into both agents' drivers and the benchmark; nisse-dev's helper loop is a follow-up. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk
This commit is contained in:
1 parent
fa8d93cf41
commit
cb638fe687
20 files changed
+952
-44
No files matched your search
@@ -2,7 +2,8 @@
|
||||
tool, runs out of turns, or is stopped.
|
||||
|
||||
Everything the run does is written to the item's transcript as the driver
|
||||
(L7), and every model call sees the driver's view of it (L6). Execution tools
|
||||
(L7), and every model call sees the driver's view of it (L6), compacted
|
||||
when it grows long (``compact``). Execution tools
|
||||
go to the item's executor, on its tool VM (L13); control tools are answered
|
||||
here, and a waiting one ends the run, so the scheduler can put it to the
|
||||
operator. Which tools wait is the agent's: each run is told.
|
||||
@@ -14,6 +15,7 @@ from typing import Any, Literal, Protocol
|
||||
|
||||
from nisse_core.log.canonical import AssistantText, Fact, ToolCall, ToolResult
|
||||
from nisse_core.loop.tools import names
|
||||
from nisse_core.models.compact import Compactor
|
||||
from nisse_core.models.llm import LLMClient, LLMError
|
||||
from nisse_core.state.cancel import Cancel, skip_calls
|
||||
from nisse_core.state.store import Transcript
|
||||
@@ -82,8 +84,10 @@ class Driver:
|
||||
max_turns: int = DEFAULT_MAX_TURNS,
|
||||
cancel: Cancel | None = None,
|
||||
failure_limit: int = 0, # 0: no limit
|
||||
compactor: Compactor | None = None,
|
||||
) -> None:
|
||||
self._client = client
|
||||
self._compactor = compactor
|
||||
self._model = model
|
||||
self._max_turns = max_turns
|
||||
self._failure_limit = failure_limit
|
||||
@@ -123,6 +127,12 @@ class Driver:
|
||||
# L2: stop takes effect at the next tool boundary.
|
||||
if self._cancel.requested:
|
||||
return RunOutcome("stopped", note=f"before turn {turn + 1}")
|
||||
if self._compactor is not None:
|
||||
# Not a turn: the model's notes replace the middle of a long
|
||||
# run before it outgrows the window (L6).
|
||||
self._compactor.compact(
|
||||
transcript, system=system, tools=tools, model=model
|
||||
)
|
||||
try:
|
||||
produced = self._client.complete(
|
||||
transcript.view(), tools=tools, model=model, system=system
|
||||
|
||||
Reference in new issue
Block a user