Flaky: test_git ssh-agent tests count every ssh-agent on the box #172

New Issue

2026-10-20T00:11:56Z

dev-agent commented

2026-10-20 00:11:56 +00:00

nisse-dev/tests/test_git.py::test_a_failed_push_still_kills_its_agent failed on main's push run for 9150c4a (the #170 merge):

AssertionError: assert {'2304'} == set()

_agents() lists every live ssh-agent on the machine (pgrep -x ssh-agent). The unit tier runs under pytest-xdist (-n auto), so another worker's push can start its own agent between before and the assertion. The runs on main before and after this one passed.

Possible fix: count only the agents this test started. One way is to match the agent's socket path, if GitRepo.push puts it somewhere the test can name, such as under tmp_path. Another is to put the ssh-agent tests in one xdist group (--dist loadgroup). Either way, add a test that fails while a second agent runs alongside.

Also seen in the same session: tests/integration/test_delegation_live.py::test_delegate_completes_a_scoped_subtask hit its 12-turn cap on claude-haiku-5-5 on #171's first CI run. That's a model-variance flake on a live test. It's noted here only to track how often it happens.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

`nisse-dev/tests/test_git.py::test_a_failed_push_still_kills_its_agent` failed on main's push run for 9150c4a (the #170 merge): ``` AssertionError: assert {'2304'} == set() ``` `_agents()` lists every live `ssh-agent` on the machine (`pgrep -x ssh-agent`). The unit tier runs under pytest-xdist (`-n auto`), so another worker's push can start its own agent between `before` and the assertion. The runs on main before and after this one passed. **Possible fix:** count only the agents this test started. One way is to match the agent's socket path, if `GitRepo.push` puts it somewhere the test can name, such as under `tmp_path`. Another is to put the ssh-agent tests in one xdist group (`--dist loadgroup`). Either way, add a test that fails while a second agent runs alongside. Also seen in the same session: `tests/integration/test_delegation_live.py::test_delegate_completes_a_scoped_subtask` hit its 12-turn cap on claude-haiku-5-5 on #171's first CI run. That's a model-variance flake on a live test. It's noted here only to track how often it happens. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01J5Utyfzw7gF7UhjRDEtjTk

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dev/nisse#172