Skip to content

fix(test): wait for the background task's working snapshot - #628

Open
ShotaroKataoka wants to merge 3 commits into
mainfrom
fix/flaky-agent-background-test
Open

ShotaroKataoka wants to merge 3 commits into
mainfrom
fix/flaky-agent-background-test

Conversation

@ShotaroKataoka

@ShotaroKataoka ShotaroKataoka commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

test_a_call_to_the_manage_tool_by_name_is_refused_and_the_task_keeps_running read the task status once, right after invoke_async, and sometimes saw ['queued'] (#626). The cause is in how Strands publishes task updates, not in the test's timing budget.

Strands' in-process engine runs on its own runtime thread and publishes every update with loop.call_soon(self._run_notification, …). The task is marked working in the first step of _execute, but that notification is queued behind the callback that hands the dispatch ack back to the agent's thread. When the GIL goes to the agent thread first, the turn (refused manage call, final text) can finish before the runtime thread stores the working snapshot in agent.state.

test_cancel_all_cancels_a_pending_task has the same single read of ["working"] and could flake the same way.

Production is not affected: session_runtime only uses background.pending(), which counts queued and working alike.

Changes

  • tests/test_agent_background.py
    • The cancel-all test polls for working with the file's _until helper (timeout 5 s) instead of reading it once.
    • The refusal test no longer asserts working. It accepts queued or working right after the turn (never cancelled) and checks that the task ends completed. It does not wait for working: the task takes 0.4 s, so it can reach completed before a poll sees working.

Testing

  • Reproduced the CI failure: a probe plugin that delays _run_notification for working by 0.1 s makes both tests on main fail with assert ['queued'] == ['working']. With this change the whole file passes under the same probe.
  • 30 runs of the two tests in a loop without the probe: 0 failures.
  • ruff check tests/test_agent_background.py

Notes

No CHANGELOG entry: test-only change.

Closes #626

🤖 Created by Kiro Crew

…of reading it once

Strands' in-process engine publishes each task update with call_soon on
its runtime thread. A task is marked working in the first step of its
execution, but that notification is queued behind the callback that
hands the dispatch ack back to the agent's thread. The agent thread can
then finish the turn before the runtime thread stores the "working"
snapshot in agent.state, and a status read right after invoke_async
sees "queued" (#626).

The two tests that read the status once now poll for "working" with the
file's _until helper. The refusal test still checks that the status
right after the turn is queued or working, never cancelled.

Production code is unaffected: session_runtime only uses
background.pending(), which counts queued and working alike.

Closes #626
The task finishes in 0.4 s, so it can reach completed before the poll ever
sees working and the wait would time out. The final completed check already
shows the refused cancel left the task running.

SPEC: 20261114-2000_pr-trio-takeover
Progress: #628 R1
Next: #634, #629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: test_agent_background.py flakes with ['queued'] == ['working']

1 participant