iOS: resolveRunnerWindow fails when an unused trailing window disappears

[wcastand]

Bug

resolveRunnerWindow(app:) eagerly reads the frame of every app window before returning the first usable one. If an unused trailing window disappears between its exists check and its frame read, XCTest records a failure even though the first/main window is still available.

Affected implementation: Agent Device 0.21.15; also still present in upstream source at 3660c7fa.

Small native reproduction

Public repo: https://github.com/wcastand/agent-device-window-race-repro
Verified commit: b1baf97

The fixture is a small UIKit app with a persistent main UIWindow. It shows a second window every 5 seconds, then hides/releases it after 600 ms. Two XCUITests repeatedly call the extracted original resolver or the proposed short-circuit version using real XCUIApplication/XCUIElement objects. No Expo/React Native, backend, account, mocked failures, or hooks/delays inside the resolver are involved.

git clone https://github.com/wcastand/agent-device-window-race-repro.git
cd agent-device-window-race-repro
./reproduce.sh <iOS-simulator-UDID>

The Xcode project is included; no package install or signing team is needed. Each loop is bounded to 60 seconds / 6,000 calls. This is a natural timing race, so a run can miss it. The script returns success only if the original produces the specific stale-element failure and the patched test passes; setup errors and missed races are rejected.

Observed vs expected

On Xcode 27.0 (27A266a), macOS 27.0 arm64, iPhone 18 Pro simulator / iOS 27.0, the script exited 0:

• Original: failed on call 332 (zero-based iteration 331).
• Patched: completed 3,843 calls over one minute, no failures.
• The original failure was also observed separately on iPhone 17 / iOS 26.5.

Sanitized native test output:

Checking existence of `Window (Element at index 1)`
Find the Window (Element at index 1)
Find the Window (Element at index 1) (retry 1)
Find the Window (Element at index 1) (retry 2)
Failed to get matching snapshot: No matches found for Element at index 1 from input {(
    Window (Main)
)}

Expected: return the first usable window without resolving later windows that cannot affect the result. The repro isolates the native resolver; it does not run the full CLI/daemon or reproduce a keyboard-specific lifecycle.

Proposed patch

Downloadable patch, checked against the upstream source path:

--- a/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift
+++ b/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift
@@ -226,12 +226,13 @@
   /// display can never name a different window than the one the reference frame was measured on.
   func resolveRunnerWindow(app: XCUIApplication) -> (window: XCUIElement?, frame: CGRect) {
     let windows = app.windows.allElementsBoundByIndex
-    var frames: [CGRect?] = []
-    for window in windows {
-      frames.append(window.exists ? window.frame : nil)
-    }
-    if let index = Self.firstUsableWindowIndex(frames: frames) {
-      return (windows[index], frames[index] ?? .zero)
+    // Stop at the first usable window. Later keyboard/overlay windows can disappear
+    // between exists and frame, recording an XCTest failure before a valid tap.
+    for window in windows where window.exists {
+      let frame = window.frame
+      if !frame.isEmpty {
+        return (window, frame)
+      }
     }
     return (nil, app.frame)
   }

This preserves first-non-empty-window selection and the app.frame fallback. It avoids unnecessary trailing-window reads; it does not make exists + frame atomic or claim to fix disappearance of the selected window.