fix: iterate (not recurse) in external dereferencing depth loop#506
Open
aqeelat wants to merge 2 commits into
Open
fix: iterate (not recurse) in external dereferencing depth loop#506aqeelat wants to merge 2 commits into
aqeelat wants to merge 2 commits into
Conversation
Components.externallyDereference implemented its depth/iteration loop recursively, nesting one large async frame per pass. For .full on a document with chained external $refs, that stacked enough multi-KB frames to overflow nightly-focal's cooperative-thread stack (intermittent SIGSEGV in test_example). It was also a latent scalability bug for deep external-ref chains. Convert the recursive self-call to a while loop by extracting one pass into a performExternalDereferencePass helper. Per-pass stack cost goes from O(passes) recursive frames to O(1). Behavior-identical; non-breaking. Applied to both OpenAPIKit and OpenAPIKit30. Fixes mattpolzin#505.
Adds a test with a cyclic external-ref graph (A→B→A) to prove that externallyDereference(with: .full) converges — the merge-collision (detectCollision keeps the old component) discards re-loaded duplicates and the loop terminates once all refs are internal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #505.
Problem
Components.externallyDereference(with:depth:)implemented its depth/iteration loop recursively, nesting one largeasyncframe per pass (each frame: 11OrderedDictionarysnapshots + 11async letfutures + 11 resolvedComponents, all live acrossawaits). For.fullon a document with chained external$refs, that stacked ~5 multi-KB frames and intermittently overflowednightly-focal's cooperative-thread stack → SIGSEGV inExternalDereferencingDocumentTests.test_example. It was flaky (Swift's task scheduler sometimes runs child tasks inline, deepening the stack) and also a latent scalability bug (deep external-ref chains would overflow even on macOS). See #505 for the full diagnosis.Fix
Convert the recursive self-call to a
whileloop by extracting one pass into aperformExternalDereferencePasshelper. Per-pass stack cost goes from O(passes) recursive frames → O(1), eliminating the depth-based stack growth.OpenAPIKitandOpenAPIKit30(identical pattern).Notes
.fullconverges for cycles via merge-collision (detectCollisionkeeps the already-resolved component). A cyclic test (A→B→A) is included to document this. No explicit cycle guard needed for loaders that follow thecomponentKeycontract.This PR was drafted with assistance from AI tooling. The submitter has reviewed and validated the contents prior to submission.