Skip to content

Enable VS integration tests on CI#19930

Open
abonie wants to merge 27 commits into
dotnet:mainfrom
abonie:enable-integration-tests
Open

Enable VS integration tests on CI#19930
abonie wants to merge 27 commits into
dotnet:mainfrom
abonie:enable-integration-tests

Conversation

@abonie

@abonie abonie commented Jun 10, 2026

Copy link
Copy Markdown
Member

WIP Testing on CI

@github-actions

Copy link
Copy Markdown
Contributor

✅ No release notes required

Can't use `dotnet` due to a conflict with mtp/xunit3
@github-actions github-actions Bot added ⚠️ Affects-Test-Tooling Tooling check: PR touches test framework infrastructure ⚠️ Affects-Build-Infra Tooling check: PR touches build infrastructure ⚠️ Affects-Restore Tooling check: PR touches NuGet packages or feeds labels Jun 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Tooling Safety Check — Affects-Build-Infra, Affects-Restore, Affects-Test-Tooling
Affects-Build-Infra: modifies azure-pipelines-PR.yml, eng/Build.ps1, eng/Versions.props
Affects-Restore: adds PackageDownload for xunit.runner.console in .csproj
Affects-Test-Tooling: new TestUsingXUnitConsole runner bypassing dotnet test

Generated by PR Tooling Safety Check · opus46 3.6M ·

abonie and others added 20 commits June 10, 2026 18:14
The two GoToDefinitionTests [IdeFact]s were no-ops: after Shell.ExecuteCommandAsync(GotoDefn)
the caret line was sampled immediately, but FSharpNavigation.NavigateToItem schedules the
actual caret move asynchronously (via JoinableTaskFactory). The previous fixed Task.Delay
then re-anchored the caret before navigation landed, making it bounce between the call site
and the definition without ever being detected.

Replace the fixed delay with GoToDefinitionWithRetryAsync: wait for the project system, anchor
the caret on the call site, invoke GotoDefn, then poll for the caret line to change (letting the
async navigation land) before treating the attempt as a no-op and retrying. The caret is only
re-anchored at the start of each attempt, never while a navigation may be in flight.

Add EditorInProcess.ActivateAsync to keep the editor as the active command context after
PlaceCaretAsync's dte.Find shifts VS's active selection (otherwise GotoDefn routes to the wrong
target and returns E_FAIL).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI showed FsiAndFsFilesGoToCorrespondentDefinitions now passes but GoesToDefinition still
no-ops (GotoDefn never resolves the symbol after 20 attempts, no command error). The only
structural difference is that the passing test builds the solution while GoesToDefinition only
edited the buffer via SetTextAsync. Building gives the F# checker the project's full options and
on-disk source, so the symbol resolves. Mirror the sibling test by building before navigating.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous build step left the Build Output pane as the active text view, so PlaceCaretAsync
searched the build log ('Marker add 1 not found in text: Build started...') instead of the
source. Re-open Library.fs after building to make it the active document again, mirroring
FsiAndFsFilesGoToCorrespondentDefinitions which opens the file it navigates after building.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
abonie and others added 5 commits June 13, 2026 22:32
…igation

Root cause is now understood, so the retry loop is unnecessary:
 * F# resolves the definition synchronously (FSharpNavigation.TryGoToDefinition blocks on the
   checker via gtdTask.Wait()), so the symbol is found on the first invocation once the project
   is loaded -- re-issuing the command never helped. GoesToDefinition's no-op was simply a
   missing build (an un-built project has no checker options), already fixed in the test body.
 * Only the navigation is asynchronous (NavigateToItem schedules the caret move via
   JoinableTaskFactory), so we issue GotoDefn once and wait for that single navigation to land.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removing the retry loop regressed GoesToDefinition on CI (build 1463019):
'GoToDefn did not navigate away from add 1 within 10s'. The first GotoDefn invocation
genuinely no-ops -- the Roslyn Document / F# checker is not ready for the freshly
rebuilt+reopened file -- and returns without navigating, so waiting longer on that single
call cannot help; the command must be RE-ISSUED. Restore the retry loop (which already waits
for the async navigation after each invocation) and document why re-issuing is load-bearing so
it is not removed again.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replaces the retry loop (which could mask a real GoToDefinition regression by eventually
succeeding) with: wait for the project system, wait a fixed FSharpCheckerSettleDelay (10s) for
the F# checker to typecheck the just-opened/built document, then issue GotoDefn ONCE and wait
for the single async navigation to land. A genuine no-op now fails the test instead of being
retried away. FSharpCheckerSettleDelay may need tuning based on CI timing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI (build 1463623) confirmed the asymmetry: FsiAndFs (adds files to disk, opens them fresh)
passes a single GotoDefn, while GoesToDefinition (SetTextAsync on the auto-opened buffer) no-ops
on the first invocation even after a 10s settle. Editing an already-open buffer is the
difference, not timing.

So restructure GoesToDefinition to mirror FsiAndFs: write the code to Test.fs, build, open it
fresh, then navigate. With a fresh document a single GotoDefn resolves, so remove both the retry
loop and the fixed settle -- the only remaining wait observes the one async navigation landing,
and a genuine no-op now fails the test instead of being retried/slept away. Also drop the
redundant standalone PlaceCaretAsync calls (GoToDefinitionAsync already places the caret).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚠️ Affects-Build-Infra Tooling check: PR touches build infrastructure ⚠️ Affects-Restore Tooling check: PR touches NuGet packages or feeds ⚠️ Affects-Test-Tooling Tooling check: PR touches test framework infrastructure

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

1 participant