Repository navigation
Python: write and read chat history files as UTF-8 - #14745
Open
Hyacinth-of-Security (lxcxjxhx) wants to merge 1 commit into
Open
Hyacinth-of-Security (lxcxjxhx) wants to merge 1 commit into
Hyacinth-of-Security (lxcxjxhx) wants to merge 1 commit into
Conversation
Hyacinth-of-Security (lxcxjxhx)
requested a review
from a team
as a code owner
November 7, 2026 18:18
Hyacinth-of-Security (lxcxjxhx)
deployed
to
github-app-auth
November 7, 2026 18:18 — with
GitHub Actions
Active
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Existing tests would still pass if explicit UTF-8 encodings were removed from ONNX initialization and chat-history persistence, so targeted regression coverage is needed.
2 open issues
What changed in this PR
This PR makes file encoding explicit in the Python package, improving chat-history portability and ensuring ONNX model configuration files are read as UTF-8.
Changes:
- Use UTF-8 when writing and reading chat-history files.
- Use UTF-8 when reading ONNX
genai_config.jsonfiles.
| File | Description |
|---|---|
python/semantic_kernel/contents/chat_history.py |
Specifies UTF-8 for history file reads and writes. |
python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_completion_base.py |
Specifies UTF-8 for model configuration reads. |
Review effort: Balanced
| try: | ||
| json_gen_ai_config = os.path.join(ai_model_path + "/genai_config.json") | ||
| with open(json_gen_ai_config) as file: | ||
| with open(json_gen_ai_config, encoding="utf-8") as file: |
| """ | ||
| json_str = self.serialize() | ||
| with open(file_path, "w") as local_file: | ||
| with open(file_path, "w", encoding="utf-8") as local_file: |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
Three text-mode
open()calls in shipped code rely on the platform default encoding. On Windows that is the ANSI code page (GBK on a zh-CN machine, cp1252 on en-US), so:ChatHistory.store_chat_history_to_file()writes a file that is not UTF-8: a history containinghéllo — 世界lands on disk ash\xa8\xa6llo \xa1\xaa \xca\xc0\xbd\xe7— GBK bytes for the same text. That file cannot be read on Linux/macOS or inside a container, whereopen()defaults to UTF-8:UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8, and any character outside the ANSI code page raisesUnicodeEncodeErrorwhen writing.ChatHistory.load_chat_history_from_file()is the matching half, so the pair only round-trips on the machine that wrote it.OnnxGenAICompletionBase._load_genai_settingsreadsgenai_config.jsonfrom a model directory without an encoding — those configs come from the model package and are UTF-8.Change
Three lines, no behaviour change on POSIX (UTF-8 is already the default there).
Verification
Windows 11 / CPython 3.13,
semantic_kernelinstalled editable frompython/:python/tests/unit/contents/test_chat_history.py: 76 passed before and after (the suite does not pin an encoding, so it does not notice the difference — the fix is about interoperability, not about a failing test).Scope
Only
python/semantic_kernel/. A scan of the tree finds 32 more text-mode opens without an encoding, but they are in tests, samples and scripts; I kept this PR to the three in the shipped package so it stays reviewable in one pass. Happy to send the test/sample sweep separately if you want it.