Add the option to run UI tests with the parallel frontend by ywxt · Pull Request #153801 · rust-lang/rust

ywxt · 2026-03-13T02:24:03Z

This PR adds two arguments for tests:

--parallel-frontend-threads: specify -Zthread to compile test case (currently UI tests only)
--iteration-count: the number of times to run each test

Also, due to the non-deterministic diagnostic orders and cycle errors, this PR adds the directive //@ ignore-parallel-frontend to ignore tests with cycle error when the parallel-frontend is enabled (by --parallel-frontend-threads) and enables //@ compare-output-by-lines by default.

Context: #t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬

This PR should work with #153797 together.

ywxt · 2026-03-13T08:11:00Z

cc @jieyouxu @Kobzol @Zoxc @zetanumbers @Ozzy1423 @teor2345

petrochenkov · 2026-03-13T11:16:50Z

This may be a bit silly, but to avoid many unnecessary changes in stderr files you could put the //@ ignore-parallel-frontend into the first empty lines in tests, like I did with //@ check-pass in #151130 (the directives shouldn't necessarily be at the top).
Hopefully the ignores will be removed away sooner or later, so this way we'll avoid thrashing the files on both directive addition and removal. (Now majority of the diff is meaningless stderr file changes.)

petrochenkov · 2026-03-13T11:19:12Z

What exactly failures are ignored by ignore-parallel-frontend in practice?
Are they .stderr file comparison errors, or there are //~ ERROR annotation mismatches too?

Right now ignore-parallel-frontend ignores the test entirely, but maybe we could ignore only some of the checks.

ywxt · 2026-03-13T11:35:02Z

What exactly failures are ignored by ignore-parallel-frontend in practice? Are they .stderr file comparison errors, or there are //~ ERROR annotation mismatches too?

Right now ignore-parallel-frontend ignores the test entirely, but maybe we could ignore only some of the checks.

failures would produce different errors so that //~ ERROR annotation mismatches

src/doc/rustc-dev-guide/src/tests/compiletest.md

tests/ui/async-await/mutually-recursive-async-impl-trait-type.rs

src/tools/compiletest/src/common.rs

ywxt · 2026-03-13T13:10:15Z

diff of stderr:

105     LL | pub static R1: &[()] = unsafe { from_ptr_range(ptr::null()..ptr::null()) }; // errors inside libcore
106        |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ evaluation of `R1` failed here
107
-       error[E0080]: in-bounds pointer arithmetic failed: attempting to offset pointer by 8 bytes, but got ALLOC10 which is only 4 bytes from the end of the allocation
109       --> $DIR/forbidden_slices.rs:54:25
110        |
111     LL |     from_ptr_range(ptr..ptr.add(2)) // errors inside libcore

157                    HEX_DUMP
158                }
159
-       error[E0080]: in-bounds pointer arithmetic failed: attempting to offset pointer by 8 bytes, but got ALLOC11+0x1 which is only 7 bytes from the end of the allocation
161       --> $DIR/forbidden_slices.rs:79:25
162        |
163     LL |     from_ptr_range(ptr..ptr.add(1))

actual:

error[E0080]: in-bounds pointer arithmetic failed: attempting to offset pointer by 8 bytes, but got alloc66+0x1 which is only 7 bytes from the end of the allocation
  --> /home/ywxt/Projects/Rust/rust/tests/ui/const-ptr/forbidden_slices.rs:79:25
   |
LL |     from_ptr_range(ptr..ptr.add(1))
   |                         ^^^^^^^^^^ evaluation of `R8` failed here

error[E0080]: in-bounds pointer arithmetic failed: attempting to offset pointer by 8 bytes, but got alloc69 which is only 4 bytes from the end of the allocation
  --> /home/ywxt/Projects/Rust/rust/tests/ui/const-ptr/forbidden_slices.rs:54:25
   |
LL |     from_ptr_range(ptr..ptr.add(2)) // errors inside libcore
   |                         ^^^^^^^^^^ evaluation of `R2` failed here

The difference is from alloc ids. Should we mask all of them?

petrochenkov · 2026-03-13T13:38:04Z

It may be possible to normalize them instead of ignoring.
I'd ping the author of the test and ask their opinion.

ywxt · 2026-03-13T16:46:46Z

tests/ui/const-generics/issues/issue-100313.rs

@@ -1,5 +1,5 @@
 //@ dont-require-annotations: NOTE
-
+//@ ignore-parallel-frontend different alloc ids


I will add normalization rules if it's confirmed.

ywxt · 2026-03-13T16:48:30Z

tests/ui/const-generics/min_const_generics/invalid-patterns.rs

 //@ stderr-per-bitwidth
 //@ dont-require-annotations: NOTE
-
+//@ ignore-parallel-frontend different alloc ids


Although some consts tests didn't get failed in my environment, I marked them as ignored due to alloc ids in stderr.

ywxt · 2026-03-13T16:50:38Z

tests/ui/lint/rfc-2383-lint-reason/force_warn_expected_lints_fulfilled.rs

 //@ compile-flags: --force-warn unused_mut
 //@ check-pass
-
+//@ ignore-parallel-frontend the message `requested on the ...` appears in different lines


The message is reported on two different source lines. Maybe we can solve it by comparison approaches?

ywxt · 2026-03-13T16:57:56Z

tests/ui/dep-graph/dep-graph-caller-callee.rs

 //@ compile-flags: -Z query-dep-graph
-
+//@ ignore-parallel-frontend dep graph
 #![feature(rustc_attrs)]


The dep graph tests failed on some rustc_then_this_would_need attributes.

Do you know the detail? @zetanumbers @Zoxc

I was able to reproduce a failure on main. It seems to be non-deterministic. Perhaps #152621 helps. I haven't looked into the issues with -Z query-dep-graph.

src/doc/rustc-dev-guide/src/tests/compiletest.md

Kobzol · 2026-03-13T21:01:14Z

src/doc/rustc-dev-guide/src/tests/compiletest.md

+
+The parallel frontend is avaliable in UI tests only at the moment, and is not currently supported in other test suites.
+
+If you run with `--parallel-frontend-threads` and `--bless`, then `--parallel-frontend-threads` will be valied.


What does this sentence mean? :) I didn't understand it due to the last word probably being typoed.

yea, this is a typo. :)

I still wouldn't understand what "would be invalid" means, tbh 😅 Let's say something like this:

If you run with both `--parallel-frontend-threads` and `--bless`, then `--parallel-frontend-threads` would be ignored, to ensure that the blessed output is deterministic.

src/tools/compiletest/src/rustdoc_gui_test.rs

ywxt · 2026-03-14T05:52:10Z

@rustbot ready

petrochenkov · 2026-03-14T17:31:05Z

src/tools/compiletest/src/common.rs

+    /// Note that the parallel frontend is disabled when `bless` is true,
+    /// to avoid swapping errors it can cause in snapshot updates.
+    pub fn parallel_frontend_enabled(&self) -> bool {
+        !self.bless && self.parallel_frontend_threads != 1


I don't think --bless and --parallel-frontend-threads should affect each other in any way.

Blessing is not used on CI, locally you can either not use blessing when running parallel frontend, or actually use it for debugging the differences in diagnostics order.

One possibility is to bless without parallel, then bless with parallel, to minimise the number of differences committed to git due to line re-ordering.

Context and longer discussion here:
#153797 (comment)

Or we may make a panic when both --bless and --parallel-frontend-threads passed?

Or we may make a panic when both --bless and --parallel-frontend-threads passed?

Why? Blessing in parallel mode is a useful functionality, even if the results are never committed.

rustbot · 2026-03-14T17:32:34Z

Some changes occurred in src/tools/compiletest

cc @jieyouxu

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @tshepang

zetanumbers · 2026-03-16T09:41:25Z

Just as I've said before in #t-compiler/parallel-rustc > Add the parallel front-end test suite @ 💬, I would like to see individual parallel rustc tests to be done sequentially. This way each rustc process would utilize CPU cores without much of context switching and as I believe reaching better parallel front-end coverage.

petrochenkov · 2026-03-16T10:16:42Z

r=me after removing the bless condition from parallel_frontend_enabled and updating the dev guide accordingly.

All the possible improvements like #153801 (comment) and #153801 (comment) can be done once the general support is merged and the tests start running on CI.

ywxt · 2026-03-16T11:56:56Z

---- [ui] tests/ui/consts/const-eval/issue-50814-2.rs#normal stdout ----
Saved the actual stderr to `/root/rust/rust/build/x86_64-unknown-linux-gnu/test/ui/consts/const-eval/issue-50814-2.normal/issue-50814-2.normal.stderr`
diff of stderr:

30      LL |     println!("{:x}", foo::<()>() as *const usize as usize);
31         |                      ^^^^^^^^^^^
32
+       note: the above error was encountered while instantiating `fn std::rt::lang_start::<()>`
+
33      error: aborting due to 1 previous error
34
35      For more information about this error, try `rustc --explain E0080`.

---- [ui] tests/ui/abi/simd-abi-checks-avx.rs stdout ----
Saved the actual stderr to `/root/rust/rust/build/x86_64-unknown-linux-gnu/test/ui/abi/simd-abi-checks-avx/simd-abi-checks-avx.stderr`
diff of stderr:

92      LL |         in_closure()();
93         |         ^^^^^^^^^^^^^^
94
+       note: the above error was encountered while instantiating `fn std::rt::lang_start::<()>`
+
95      error: aborting due to 11 previous errors
96
97

some new failures under the parallel frontend due to the new note note: the above error was encountered while instantiating fn std::rt::lang_start::<()>`. Anyone knows it?

failures:
    [ui] tests/ui/abi/simd-abi-checks-avx.rs
    [ui] tests/ui/consts/const-eval/issue-50814-2.rs#mir-opt
    [ui] tests/ui/consts/mono-reachable-invalid-const.rs
    [ui] tests/ui/consts/required-consts/collect-in-called-fn.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-called-fn.rs#opt
    [ui] tests/ui/consts/required-consts/collect-in-dead-closure.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-closure.rs#opt
    [ui] tests/ui/consts/required-consts/collect-in-dead-drop.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fn-behind-assoc-type.rs#opt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fn-behind-generic.rs#opt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fn-behind-assoc-type.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fn.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fnptr-in-const.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fnptr.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-fnptr.rs#opt
    [ui] tests/ui/consts/required-consts/collect-in-dead-move.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-promoted-const.rs#noopt
    [ui] tests/ui/consts/required-consts/collect-in-dead-move.rs#opt
    [ui] tests/ui/lint/large_assignments/inline_mir.rs
    [ui] tests/ui/lint/large_assignments/move_into_fn.rs
    [ui] tests/ui/lint/large_assignments/copy_into_box_rc_arc.rs
    [ui] tests/ui/lint/large_assignments/copy_into_fn.rs
    [ui] tests/ui/lint/large_assignments/large_future.rs#attribute
    [ui] tests/ui/lint/non-snake-case/lint-uppercase-variables.rs
    [ui] tests/ui/rust-2018/edition-lint-inter-outlives/edition-lint-infer-outlives-macro.rs
    [ui] tests/ui/simd/const-err-trumps-simd-err.rs
    [ui] tests/ui/structs/default-field-values/post-mono.rs#indirect

Zoxc · 2026-03-16T12:03:54Z

@ywxt Sounds like #85633, which does have race conditions

petrochenkov · 2026-03-16T12:06:14Z

@ywxt Sounds like #85633, which does have race conditions

I first thought about #153194, but it's not even merged yet :D

ywxt · 2026-03-16T12:35:16Z

@ywxt Sounds like #85633, which does have race conditions

I think it should be a new commit. I didn't find it before at all.

This comment has been minimized.

Sign in to view

petrochenkov self-assigned this Mar 13, 2026

ywxt force-pushed the parallel-test branch from e95bfb4 to 6d401a3 Compare March 13, 2026 06:43

petrochenkov added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 13, 2026

petrochenkov reviewed Mar 13, 2026

View reviewed changes

src/doc/rustc-dev-guide/src/tests/compiletest.md Outdated Show resolved Hide resolved

tests/ui/async-await/mutually-recursive-async-impl-trait-type.rs Outdated Show resolved Hide resolved

src/tools/compiletest/src/common.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 13, 2026

ywxt force-pushed the parallel-test branch 2 times, most recently from 67ebcd0 to 414d715 Compare March 13, 2026 16:44

ywxt commented Mar 13, 2026

View reviewed changes

Kobzol reviewed Mar 13, 2026

View reviewed changes

ywxt force-pushed the parallel-test branch from 414d715 to d9c6a67 Compare March 14, 2026 02:48

ywxt added 2 commits March 14, 2026 11:04

Run ui tests on the parallel frontend

ad4bc8b

Ignore tests for the parallel frontend

5aeb919

ywxt force-pushed the parallel-test branch from d9c6a67 to 5aeb919 Compare March 14, 2026 03:05

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Mar 14, 2026

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 14, 2026

petrochenkov reviewed Mar 14, 2026

View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 14, 2026

petrochenkov marked this pull request as ready for review March 14, 2026 17:32

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 14, 2026

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Mar 14, 2026

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 14, 2026

Kobzol changed the title ~~Run ui tests on the parallel frontend~~ Add the option to run UI tests with the parallel frontend Mar 16, 2026

		@@ -1,5 +1,5 @@
		//@ dont-require-annotations: NOTE

		//@ ignore-parallel-frontend different alloc ids


		The parallel frontend is avaliable in UI tests only at the moment, and is not currently supported in other test suites.

		If you run with `--parallel-frontend-threads` and `--bless`, then `--parallel-frontend-threads` will be valied.

Uh oh!

Conversation

ywxt commented Mar 13, 2026

Uh oh!

This comment has been minimized.

ywxt commented Mar 13, 2026

Uh oh!

petrochenkov commented Mar 13, 2026

Uh oh!

petrochenkov commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ywxt commented Mar 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ywxt commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ywxt commented Mar 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rustbot commented Mar 14, 2026

Uh oh!

zetanumbers commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Mar 16, 2026

Uh oh!

ywxt commented Mar 16, 2026

Uh oh!

Zoxc commented Mar 16, 2026

Uh oh!

petrochenkov commented Mar 16, 2026

Uh oh!

ywxt commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

petrochenkov commented Mar 13, 2026 •

edited

Loading

ywxt commented Mar 13, 2026 •

edited

Loading

petrochenkov commented Mar 13, 2026 •

edited

Loading

zetanumbers commented Mar 16, 2026 •

edited

Loading

ywxt commented Mar 16, 2026 •

edited

Loading