dongarandClaude Opus 5.5 ad8a07fcb7 Wait out LLM outages with backoff (L5: outages delay, never fail) ...

A 502, 503 or 504 from the LiteLLM proxy, or a dropped connection, is
retried after 5, 15 and 45 seconds before the call fails. Twice on
2026-10-07 a short burst of 502s stopped work outright: the integration
tier at 16:39 and an e2e conversation at 20:43, where the item waited on
the operator after one refused call.

Not retried: a 500 (it may be the request's fault), a timeout (it already
had its request_timeout_s), or a budget refusal. Rate limits keep their
single 20-second retry. The time spent waiting is reported with the rate
limit's (last_rate_limit_wait_s), so it isn't counted as the model's.

Tests: an outage waited out, one that outlasts the backoff, a dropped
connection, a 500 and a timeout not retried, a budget refusal not waited
on. The offline agent in test_agent_build skips the backoff (its proxy
doesn't exist).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPQ14L3LikyefCMFBqEbKE

2026-10-07 21:11:45 +00:00

..

model_contract.py

Fix the run-breaking tool-result order; e2e test runs the slice (M2.1, PR D)

2026-10-03 15:11:06 +00:00

test_abandon.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_agent_build.py

Wait out LLM outages with backoff (L5: outages delay, never fail)

2026-10-07 21:11:45 +00:00

test_approval_binding.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_approvals.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_bench.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_canonical.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_catalog.py

Rename the delegate and consultant roles to helper and expert

2026-10-01 23:04:37 +00:00

test_config.py

Roles, capability profiles and help on the new loop (M2.6)

2026-10-07 16:33:26 +00:00

test_context.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_control.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_delegation.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_dev_main.py

Make the config file the only home for settings

2026-09-23 23:25:58 +00:00

test_dump.py

Give each role its own view of the transcript (M1.2)

2026-10-02 00:39:38 +00:00

test_evals.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_execution.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_executor.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_expert.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_gates.py

Retire the old build spec, and point every reference at the architecture (M0.1)

2026-10-01 00:17:48 +00:00

test_git.py

Keep credentials out of reprs; count only live ssh-agents in tests

2026-10-02 22:41:01 +00:00

test_gitea.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_llm.py

Wait out LLM outages with backoff (L5: outages delay, never fail)

2026-10-07 21:11:45 +00:00

test_loop_conversations.py

Roles, capability profiles and help on the new loop (M2.6)

2026-10-07 16:33:26 +00:00

test_loop_help.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_loop_labels.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_loop_milestones.py

Roles, capability profiles and help on the new loop (M2.6)

2026-10-07 16:33:26 +00:00

test_loop_multi.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_loop_new_projects.py

New projects from conversation (M2.5, PR B)

2026-10-07 15:39:00 +00:00

test_loop_projects.py

Projects as configuration entries (M1.4, D8)

2026-10-06 01:12:57 +00:00

test_loop_properties.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_loop_queue.py

Multi-milestone work and documents on the new loop (M2.4)

2026-10-05 20:57:37 +00:00

test_loop_talk.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_loop.py

Conversation on the new loop (M2.7)

2026-10-07 18:01:51 +00:00

test_matrix.py

Matrix threads in the transport (M2.1, PR A)

2026-10-02 23:40:24 +00:00

test_no_stranding.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_oplog.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_planner.py

Give each role its own view of the transcript (M1.2)

2026-10-02 00:39:38 +00:00

test_poller.py

Retire the old build spec, and point every reference at the architecture (M0.1)

2026-10-01 00:17:48 +00:00

test_pr_open_handling.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_pr_open.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_probe.py

Enforce the tool-call contract, asking servers only where it works

2026-09-24 22:54:49 +00:00

test_publish.py

Publish only the work: leave build artifacts out and report them (#80)

2026-10-02 22:38:13 +00:00

test_quint_counterexamples.py

Flatten harness branch names so a model's branch can't block them

2026-10-02 22:29:20 +00:00

test_reconcile.py

Implement A9 state-change logging in the state store

2026-09-05 21:13:36 +00:00

test_render_anthropic.py

Add the Anthropic passthrough renderer and client (A10/A13) — 7a

2026-09-07 23:13:22 +00:00

test_render.py

Retire the old build spec, and point every reference at the architecture (M0.1)

2026-10-01 00:17:48 +00:00

test_review_loop.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_state.py

Implement A9 state-change logging in the state store

2026-09-05 21:13:36 +00:00

test_stop.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_tools.py

Run every command on the build VM over SSH (M1.3)

2026-10-02 21:39:46 +00:00

test_views.py

Fix the run-breaking tool-result order; e2e test runs the slice (M2.1, PR D)

2026-10-03 15:11:06 +00:00

test_work_items.py

Put work items in the state store, and bind the log to them (M1.1)

2026-10-01 23:20:29 +00:00

test_worker.py

Add the operator kill switch, stop (A4)

2026-09-23 22:47:29 +00:00