Python: Read and write chat history files as UTF-8 - #14445
Python: Read and write chat history files as UTF-8#14445Kumaresan-AI-Engineer wants to merge 1 commit into
Conversation
ChatHistory.store_chat_history_to_file and load_chat_history_from_file opened the file in text mode without an encoding, so they used the locale encoding. On Windows that is cp1252, and storing a history with non-ASCII content raised UnicodeEncodeError; a history written as UTF-8 elsewhere also failed to load. Both now pass encoding="utf-8", matching how the rest of the package reads text files (KernelPlugin.from_directory and KernelFunctionFromPrompt.from_directory default to utf-8). Adds a regression test that round-trips a history with Chinese, accented and emoji content, which fails on the Windows CI legs before this change.
|
Kumaresan-AI-Engineer please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
🟢 Approval recommended
The UTF-8 changes are covered by regression testing, with no unresolved review comments.
Pull request overview
Updates chat history file I/O to use UTF-8 consistently across platforms.
Changes:
- Uses UTF-8 when reading and writing chat history files.
- Adds regression coverage for non-ASCII content.
File summaries
| File | Summary |
|---|---|
python/tests/unit/contents/test_chat_history.py |
Tests non-ASCII chat history round-tripping. |
python/semantic_kernel/contents/chat_history.py |
Applies UTF-8 encoding to file reads and writes. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Motivation and Context
ChatHistory.store_chat_history_to_file and load_chat_history_from_file were opening files in text mode without specifying an encoding. This meant Python used the system's default locale encoding, which is typically cp1252 on Windows.
As a result, saving chat history containing non-ASCII characters (such as Chinese characters, accented characters, or emojis) could raise a UnicodeEncodeError. Similarly, chat history files written as UTF-8 could fail to load correctly on Windows.
This change makes both methods explicitly use UTF-8, which is also consistent with how other parts of the package handle text files, such as KernelPlugin.from_directory and KernelFunctionFromPrompt.from_directory.
Description
Explicitly use encoding="utf-8" when saving chat history.
Explicitly use encoding="utf-8" when loading chat history.
Added a regression test that saves and loads chat history containing Chinese, accented, and emoji characters.
The regression test ensures that chat history can be safely round-tripped across platforms, including Windows.