[Qiming Bao (14H034160212)]

Motivation and Context

Fixes #14735

ONNXTemplate.PHI3V and ONNXTemplate.PHI4MM built prompts that the Phi-3-vision, Phi-3.5-vision and Phi-4-multimodal models were not trained on:

• <|image_N|> / <|audio_N|> were written outside the <|user|> ... <|end|> turn. With [text, image] the tag came after <|end|>, and with [image, text] it came before <|user|>.
• Each TextContent item opened its own <|user|> turn.

before: '<|user|>\nDescribe this image.<|end|>\n<|image_1|>\n<|assistant|>\n'
after:  '<|user|>\n<|image_1|>\nDescribe this image.<|end|>\n<|assistant|>\n'

The new format is the one given in the Phi-3-vision, Phi-3.5-vision and Phi-4-multimodal model cards, and it is how the onnxruntime-genai multimodal example builds the user content (get_user_content): image tags, then audio tags, then the text.

Description

• Added a private helper, _multimodal_user_content, in connectors/ai/onnx/utils.py. For one user message it returns the image placeholders, then the audio placeholders (Phi-4-mm only), each on its own line, then the text items joined with \n, along with the updated image and audio counters. Placeholders are still numbered across the whole chat history, starting at 1. That numbering matches the image and audio lists that _get_images_from_history / _get_audios_from_history pass to the processor, and Phi-3-vision's requirement that image ids be consecutive from 1.
• phi3v_template and phi4mm_template now emit a single <|user|>\n{content}<|end|>\n per user message. System and assistant turns, and the \n style, are unchanged.
• Removed the outdated "Currently only one image is supported in Onnx" comment. The onnxruntime-genai Phi-4-mm example passes several images.
• Tests (tests/unit/connectors/ai/onnx/services/test_onnx_utils.py): the two existing multimodal tests asserted the broken format, so I updated them. I also added cases for [image, text], multiple images, image numbering across turns, several text items in one message, text + audio, audio only, and audio listed before images.

No public API changes. Only the prompt string produced for multimodal user messages changes.

Verification (from python/):

uv run --no-sync pytest tests/unit/connectors/ai/onnx            # 43 passed (11 of the new/updated tests fail on main)
uv run --no-sync ruff check <changed files> && uv run --no-sync ruff format --check <changed files>
uv run --no-sync mypy -p semantic_kernel.connectors.ai.onnx --config-file mypy.ini   # Success: no issues found
uv run --no-sync pre-commit run -c python/.pre-commit-config.yaml --files <changed files>  # all hooks passed

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[Copilot]

🟢 Approval recommended

No active review issues are reported for the multimodal prompt-formatting changes.

No open issues 🎉

What changed in this PR

This PR fixes multimodal prompt formatting in the Python ONNX connector for Phi-3-vision and Phi-4-multimodal, addressing the media-placement portion of #14735.

Changes:

• Places image and supported audio placeholders inside a single user turn, before text.
• Shares content formatting between templates while preserving media numbering across chat history.
• Updates and expands regression tests for content ordering, multiple images, multiple text items, and audio-only messages.

File	Description
python/​tests/​unit/​connectors/​ai/​onnx/​services/​test_onnx_utils.py	Corrects expected prompts and adds multimodal formatting regression cases.
python/​semantic_kernel/​connectors/​ai/​onnx/​utils.py	Adds a shared helper and updates both templates to emit one correctly formatted user turn per message.

_{Review effort: Balanced}