Skip to content

openai-v2: await AsyncStream.close on default async streaming path#4751

Merged
xrmx merged 5 commits into
open-telemetry:mainfrom
yulin-li:yulin-li-fix-async-stream-close-leak
Jul 8, 2026
Merged

openai-v2: await AsyncStream.close on default async streaming path#4751
xrmx merged 5 commits into
open-telemetry:mainfrom
yulin-li:yulin-li-fix-async-stream-close-leak

Conversation

@yulin-li

Copy link
Copy Markdown
Contributor

Description

BaseStreamWrapper.close() is synchronous and calls self.stream.close() without awaiting it. On the default (non-experimental) async streaming path, async_chat_completions_create_v_old returns a LegacyChatStreamWrapper wrapping an openai.AsyncStream, whose close() is a coroutine. Calling it without await therefore:

  1. emits RuntimeWarning: coroutine 'AsyncStream.close' was never awaited, and
  2. never closes the underlying httpx response/connection, leaking it on every early-closed async stream.

This also means a correct caller doing await wrapper.close() got None back, so they could not close the real stream either.

The experimental path (gen_ai_latest_experimental opt-in) was already fixed by #4500 via util-genai's AsyncStreamWrapper, but the default path remained affected.

This PR adds AsyncLegacyChatStreamWrapper(LegacyChatStreamWrapper) with an awaitable close() that awaits the underlying AsyncStream.close(), and uses it for async streaming chat completions on the legacy path. This mirrors the existing sync/async split already present in response_wrappers.py (AsyncResponseStreamWrapper).

Fixes #4710

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Added test_async_chat_completion_streaming_close_awaits_underlying_stream, which asserts the legacy async wrapper's close() awaits the underlying AsyncStream.close() and finalizes the span.
  • Updated test_async_chat_completion_streaming_not_complete to await response.close() on the legacy path.
  • Ran the openai-v2 suite with -W error::RuntimeWarning; the coroutine ... was never awaited warning is gone and all tests pass.

Does This PR Require a Core Repo Change?

  • Yes.
  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@yulin-li yulin-li requested a review from a team as a code owner June 27, 2026 05:59
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 27, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: Copilot / name: Copilot (958c563)
  • ✅ login: yulin-li / name: Yulin Li (958c563)

@github-actions github-actions Bot added the gen-ai Related to generative AI label Jun 27, 2026
@github-actions github-actions Bot requested a review from lmolkova June 27, 2026 05:59
@yulin-li yulin-li force-pushed the yulin-li-fix-async-stream-close-leak branch 3 times, most recently from 4e7d3e1 to 958c563 Compare June 27, 2026 16:48
BaseStreamWrapper.close is synchronous and calls self.stream.close()
without awaiting it. On the default (non-experimental) async streaming
path the wrapped stream is an openai.AsyncStream whose close() is a
coroutine, so the underlying httpx response/connection was never closed
(leak) and a "coroutine ... was never awaited" RuntimeWarning was
emitted.

Add AsyncLegacyChatStreamWrapper with an awaitable close() that awaits
the underlying AsyncStream.close(), and use it for async streaming chat
completions on the legacy path. Mirrors the existing sync/async split in
response_wrappers.

Fixes open-telemetry#4710

Assisted-by: Claude Opus 4.8
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@yulin-li yulin-li force-pushed the yulin-li-fix-async-stream-close-leak branch from 958c563 to 87e6cda Compare June 27, 2026 17:05

@xrmx xrmx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to cut a new release for openai-v2 thinking this was fixed in main, but isn't. So this LGTM and I think is small enough to merge even if the work has been moved to the genai repo.

yulin-li and others added 2 commits July 4, 2026 23:40
openai-v2 is now released independently, so its changelog fragments live
in instrumentation-genai/opentelemetry-instrumentation-openai-v2/.changelog
rather than the repo root. Move 4751.fixed there per review feedback.

Assisted-by: Claude Opus 4.8
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-project-automation github-project-automation Bot moved this to Approved PRs in Python PR digest Jul 6, 2026
@yulin-li

yulin-li commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi, I don't have permission to merge, could you please help to merge it?

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a resource leak in the OpenAI v2 (legacy/default semconv) async streaming chat completions path by ensuring the wrapper’s close() properly awaits openai.AsyncStream.close(), preventing un-awaited coroutine warnings and leaked httpx responses/connections.

Changes:

  • Introduce AsyncLegacyChatStreamWrapper with an async def close() that awaits the underlying AsyncStream.close().
  • Update async streaming chat completion tests to always await response.close() on the legacy path and add a regression test for awaiting the underlying close.
  • Add a towncrier changelog entry describing the fix.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py Return an async legacy wrapper for async streaming chat completions and add an awaitable close() implementation.
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_chat_completions.py Update legacy async streaming test to await close() and add a regression test asserting the underlying AsyncStream.close() is awaited.
instrumentation-genai/opentelemetry-instrumentation-openai-v2/.changelog/4751.fixed Document the bug fix in the package changelog.

Address review feedback on the legacy async stream wrapper:

- AsyncLegacyChatStreamWrapper.__aexit__ now awaits the underlying
  AsyncStream.close so using the response as an async context manager
  (async with) releases the httpx response/connection instead of
  leaking it.
- close() finalizes the span in a finally block so the span is always
  ended even if AsyncStream.close raises, re-raising the original
  exception unmodified.
- Tests wrap the real close with AsyncMock(wraps=...) so the connection
  is still released while asserting it was awaited, and add coverage for
  the async context manager exit path.

Assisted-by: Claude Opus 4.8
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@xrmx xrmx added this pull request to the merge queue Jul 8, 2026
@xrmx xrmx removed this pull request from the merge queue due to a manual request Jul 8, 2026
@xrmx xrmx added this pull request to the merge queue Jul 8, 2026
Merged via the queue into open-telemetry:main with commit 2e75a79 Jul 8, 2026
760 checks passed
@github-project-automation github-project-automation Bot moved this from Approved PRs to Done in Python PR digest Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gen-ai Related to generative AI

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

openai-v2: BaseStreamWrapper.close() never awaits AsyncStream.close() — async streaming leaks connections/memory (OOM)

5 participants