External dereferencing intermittently crashes (SIGSEGV) on nightly-focal

[aqeelat]

Summary

Document.externallyDereference(with: .full) intermittently crashes the test suite on the nightly-focal CI job with signal 11 (SIGSEGV) — a stack overflow in ExternalDereferencingDocumentTests.test_example (ExampleLoader.load recursing through the external-dereferencing machinery).

It's flaky: identical code on the same swiftlang/swift:nightly-focal image failed one run and passed the next ~13 minutes later. It passes on every other CI job (macOS, Swift 6.2/6.3, nightly-jammy).

Root cause

Components.externallyDereference(with:depth:) implements its depth/iteration loop recursively instead of iteratively — Sources/OpenAPIKit/Components Object/Components.swift:539-544 (and the OpenAPIKit30 mirror at Components.swift:411-414):

switch depth {
case .iterations(let number):
    return try await externallyDereference(with: loader, depth: .iterations(number - 1), context: newMessages)
case .full:
    return try await externallyDereference(with: loader, depth: .full, context: newMessages)
}

Each recursive frame is a large async function (11 OrderedDictionary snapshots + 11 async let futures + 11 resolved Components, all live across awaits). For a document with chained external $refs, .full nests one such frame per pass (~5 passes for test_example). On nightly-focal's smaller cooperative-thread stacks this overflows → SIGSEGV. Flakiness comes from Swift's task scheduler sometimes running child tasks inline (deepening the stack) vs on separate workers (shallow).

This is also a latent scalability bug: a real-world spec with a deeper external-ref chain would overflow even on macOS.

Suggested fix

Convert the recursive depth loop to a while loop (the body already does exactly one pass) — O(passes) recursive frames → O(1), removing the depth-based stack growth. Non-breaking, internal-only. Needs applying to both OpenAPIKit and OpenAPIKit30.

Out-of-scope follow-ups

• Cycle detection for external refs — Update: cycles do NOT infinite-recurse. Components.merge uses detectCollision (keeps the already-resolved component on key collision), so re-loaded duplicates are discarded and .full converges once all refs are internal. Verified by a cyclic test (A→B→A) in PR fix: iterate (not recurse) in external dereferencing depth loop #506. No explicit cycle guard needed for loaders that follow the componentKey contract.
• Memoization of loaded URLs — the same URL can be loaded multiple times (e.g. string_param.json from ~7 places); caching is a perf win only (doesn't change recursion depth).

Reproduction

The existing ExternalDereferencingDocumentTests.test_example triggers it intermittently on nightly-focal. Confirmed occurrence: run 29146983087 (signal 11 in test_example); the rerun passed.

---

This issue was drafted with assistance from AI tooling. The submitter has reviewed and validated the contents prior to submission.