fix(http-client-python): make Model.__new__ lazy init thread-safe#11236
Draft
l0lawrence wants to merge 2 commits into
Draft
fix(http-client-python): make Model.__new__ lazy init thread-safe#11236l0lawrence wants to merge 2 commits into
l0lawrence wants to merge 2 commits into
Conversation
commit: |
Contributor
|
All changed packages have been documented.
Show changes
|
|
You can try these changes here
|
d7347da to
10c448d
Compare
…hout locks Concurrent first-time model construction could raise "RuntimeError: dictionary changed size during iteration" in Model.__new__, which deserialization silently swallowed, returning raw dicts instead of models. Iterate a snapshot of each class __dict__ and publish _attr_to_rest_field atomically, guarded by the class's own __dict__. Fixes #11234 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
10c448d to
4451ec6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #11234 (from Azure/azure-sdk-for-python#47426).
What was going wrong
Every generated model class builds a small table of its fields (
_attr_to_rest_field) the first time you create one of its objects, then caches it on the class.If your program creates the first objects of several related models at the same time from multiple threads (for example, an app handling its first requests from a thread pool right after startup), that first-time setup could crash with
RuntimeError: dictionary changed size during iteration— one thread was reading a parent class's attributes while another thread was writing the cache onto that same parent.The crash never surfaced, because deserialization catches any error and falls back to returning the raw JSON. So instead of an error, you'd silently get back a plain
dict(or a model with raw-dict fields inside) instead of a real model object — only on the first few responses after startup, only under concurrency. Very hard to diagnose.The fix (no locks)
Reworked the first-time setup in
Model.__new__so it's safe when several threads hit it at once:dict(mro_class.__dict__)), so another thread writing the cache can't change what we're reading mid-loop. This removes the crash. The copy has all the same fields, because a class's fields are fixed when it's defined — only cached,_-prefixed values get added later, and those are filtered out anyway._attr_to_rest_fieldis built fully in a local variable and assigned last. So the moment another thread sees it, everything is ready — no half-built state."_attr_to_rest_field" not in cls.__dict__(this class's own attributes) instead of a shared set that every class wrote into. This is correct per-class and removes another piece of unsynchronized shared state.Out of scope (follow-up)
Deserialization still silently swallows any error and returns the raw JSON. Fixing the race removes this trigger, but that broad catch would hide future errors the same way — worth a separate discussion rather than changing fallback behavior across every SDK here.