Python: gen_ai.conversation.id is unstable across turns when using Responses API

[benke520]

Python: gen_ai.conversation.id is unstable across turns when using Responses API

Issue: #5992

---

Problem

When using the Responses API (GPT-4o+, GPT-5), gen_ai.conversation.id changes on every agent.run() call, making it impossible to group multi-turn conversations in Application Insights without manual instrumentation.

Root Cause

MAF emits session.service_session_id as the gen_ai.conversation.id span attribute (via thread_id in observability.py):

# observability.py ~line 1541
thread_id=session.service_session_id if session else None

This works for Chat Completions API where service_session_id is a stable conversation handle (e.g., conv_xxx). But the Responses API uses linked-list-style response IDs — each response returns a new ID that becomes the next call's previous_response_id:

Turn 1: service_session_id = None       → LLM responds "resp_abc" → stored
Turn 2: service_session_id = "resp_abc" → gen_ai.conversation.id = "resp_abc" → LLM responds "resp_def" → stored
Turn 3: service_session_id = "resp_def" → gen_ai.conversation.id = "resp_def" → LLM responds "resp_ghi" → stored

Each invoke_agent span gets a different gen_ai.conversation.id. App Insights cannot group them as one conversation.

Impact

All Foundry users are affected. The Foundry ecosystem (FoundryChatClient, AzureAIClient) exclusively uses the Responses API — there is no Chat Completions path. This means every multi-turn conversation going through Foundry will have broken gen_ai.conversation.id grouping by default.

• Multi-turn conversations appear as disconnected transactions in App Insights — each turn gets a different gen_ai.conversation.id, making it impossible to query "show me all turns in this conversation"
• This is the default and only path for Foundry — not an edge case or opt-in behavior
• Developers must add manual parent spans (get_tracer().start_as_current_span(...)) as a workaround — but this only works in-process, not across HTTP requests (e.g., hosted agents)

Current Workaround

Wrap all turns in a manual parent span:

with get_tracer().start_as_current_span("Conversation") as span:
    session = agent.create_session()
    for question in questions:
        await agent.run(question, session=session)

This gives all turns the same operation_id (trace_id), but doesn't fix gen_ai.conversation.id.

Possible Fixes

Layer	Approach	Trade-off
Responses API	Add a stable conversation_id field	Not in MAF's control
MAF	Use session.session_id	Loses server-side correlation
MAF	Freeze first service_session_id	Stable + still server-correlated
MAF	Emit both as separate attributes	No trade-off, just more data

Environment

• agent-framework 1.4.0
• Responses API (via FoundryChatClient or OpenAI SDK)
• Application Insights / Azure Monitor

Related

• session.service_session_id is set in _agents.py (_propagate_conversation_id / _post_hook)
• Telemetry mapping in observability.py (thread_id → CONVERSATION_ID)

[chetantoshniwal]

Can you share the code sample you are running.