test: restore the TPC-H suite’s 2 GiB off-heap budget - #5904
Open
ErikBPF wants to merge 3 commits into
Open
Conversation
Current SF1 queries pass with the original budget in both native build profiles, so the 4 GiB increase is no longer needed for this suite. Refs apache#5704
github-actions
Bot
added
enhancement
andygrove
approved these changes
Sep 13, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Sep 14, 2026
andygrove
added
the
run-benchmark-check
andygrove
enabled auto-merge
September 14, 2026 13:56
Member
|
Merge failed due to flakey test (fix is in #5919). Trying again... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to #5704. This restores the previous suite setting; it does not
establish whether other workloads have changed memory requirements.
Rationale for this change
The SF1 TPC-H suite passes on current main with the original 2 GiB off-heap
budget, so the increase to 4 GiB is no longer needed for this configuration.
What changes are included in this PR?
Restore
MEMORY_OFFHEAP_SIZEfrom4gto2ginCometTPCHQuerySuite.Query plans, data generation, and expected results are unchanged.
How are these changes tested?
On base
f69c4c81b9429e327ea95658530ae4ed4ed19635, Spark 4.1.3 / Scala2.13.17 / JDK 17, the complete suite passed against repository-generated
SF1 Parquet with one partition per table, with both debug and release
native libraries: 22 tests, 66 query/join combinations, zero ignored.
SPARK_TPCH_DATA=/path/to/sf1_parquet ./mvnw -B -Prelease test \ -Dtest=none -Dsuites=org.apache.spark.sql.CometTPCHQuerySuiteThis is a suite-budget check, not a historical performance or memory
benchmark. Other Spark versions and the original DataFusion upgrade commit
were not measured.