fix(libsy): run should return when stream ends

[messiaen]

What

Algorithm::run should return when the stream has a final result canceling any in_flight CallLlm futures.

Before this MR run would continue running until all the futures were done even after the final result.

Also simplified the logic a bit and remove the arbitrary in flight cap. (no thanks to claude)

Why

• nvbug 6501297
• nvbug 6501299
• nvbug 6502618

How tested

• uv run ruff check . clean
• uv run mypy switchyard clean
• uv run pytest tests/ green
• Manual smoke (describe what was run)

Checklist

• One class per file; filename = snake_case of the primary class.
• New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use.
• Unit tests added for new components / bug fixes.
• README / --help updated if customer-facing surface changed.
• Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

Anything reviewers should pay extra attention to — risky paths, follow-up tickets, intentional trade-offs.

Summary by CodeRabbit

• Bug Fixes
  • Improved concurrent processing so results are returned promptly when multiple operations are in flight.
  • Ensured the first successful response is preserved without being overwritten by slower operations.
  • Prevented hangs when a concurrent operation never completes.
  • Terminal errors are now surfaced promptly, even during heavy concurrent activity.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 59806d72-63c9-418d-99ac-82f45ddd8a5e

📥 Commits

Reviewing files that changed from the base of the PR and between 060ad75 and 455d0da.

📒 Files selected for processing (1)

• crates/libsy/src/core/algorithm.rs

---

Walkthrough

Algorithm::run now consumes concurrent LLM calls without an internal concurrency cap and stops on terminal return or stream exhaustion. Tests cover first-wins hedging, unresolved losers, and prompt terminal-error propagation.

Changes

Algorithm execution

Layer / File(s)	Summary
Unbounded Algorithm::run loop crates/libsy/src/core/algorithm.rs	Removes the concurrency cap and guarded stream polling, allowing run to process in-flight calls while continuing to consume steps until termination.
Hedging and termination coverage crates/libsy/src/core/algorithm.rs	Adds loser and gated clients plus a Hedge algorithm, testing winner preservation, unresolved losers, and bounded terminal-error reporting.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit watching futures race,
One bright winner claims its place.
Slow gray losers hop along,
The stream keeps moving, neat and strong.
Errors thump before too long—
Carrots cheer the passing song!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: Algorithm::run now returns when the stream ends and a final result is produced.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}