Skip to content

Conversation

@raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Dec 12, 2025

Description

Remove enable-large-dynamic-filters and dynamic-filtering.small.* configuration properties since large dynamic filters have been the default for a while.
Rename dynamic-filtering.large.* configs to drop large since the small/large distinction is no longer relevant.

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## General
* {{breaking}} `enable-large-dynamic-filters` configuration property and the corresponding system session property `enable_large_dynamic_filters` has been removed. Large dynamic filters are used by default. ({issue}`27637`)
* {{breaking}} `dynamic-filtering.small*` configuration properties are now defunct and must be removed from server configurations. ({issue}`27637`)
* {{breaking}} The previously deprecated `dynamic-filtering.large-broadcast*` configuration properties are now defunct and must be removed from server configurations. ({issue}`27637`).
* `dynamic-filtering.large*` configuration properties have been deprecated in favor of `dynamic-filtering.*`. ({issue}`27637`)

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR cleans up dynamic filter configuration by removing the deprecated enable-large-dynamic-filters configuration property and dynamic-filtering.small.* settings, while renaming dynamic-filtering.large.* properties to dynamic-filtering.*. Large dynamic filters have been the default behavior for a while, making the small/large distinction unnecessary.

Key changes:

  • Removed enable-large-dynamic-filters configuration property and corresponding session property
  • Removed all dynamic-filtering.small.* configuration properties (now marked as defunct)
  • Renamed dynamic-filtering.large.* to dynamic-filtering.* with proper legacy config support
  • Simplified code by removing conditional logic based on filter size mode
  • Updated tests to use new configuration property names with appropriate limits

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
core/trino-main/src/main/java/io/trino/execution/DynamicFilterConfig.java Removed small/large filter config fields, renamed large to default, added defunct and legacy config annotations for backward compatibility
core/trino-main/src/main/java/io/trino/SystemSessionProperties.java Removed ENABLE_LARGE_DYNAMIC_FILTERS constant, session property definition, and accessor method
core/trino-main/src/main/java/io/trino/sql/planner/LocalExecutionPlanner.java Removed small filter fields, simplified methods to no longer check session for filter size mode
core/trino-main/src/main/java/io/trino/server/DynamicFilterService.java Removed small filter size limit field and conditional size limit selection logic
core/trino-main/src/test/java/io/trino/execution/TestDynamicFilterConfig.java Updated test to validate only the new configuration properties without small/large distinction
core/trino-main/src/test/java/io/trino/server/TestDynamicFilterService.java Removed session property setup and config method calls for small filters
testing/trino-tests/src/test/java/io/trino/execution/TestCoordinatorDynamicFiltering.java Updated config properties from small to default names and added explicit range row limits
testing/trino-testing/src/main/java/io/trino/testing/BaseDynamicPartitionPruningTest.java Updated config property names from large-partitioned to partitioned, removed session property setup
testing/trino-testing/src/main/java/io/trino/testing/AbstractDistributedEngineOnlyQueries.java Removed session builder setup for large dynamic filters in test
testing/trino-faulttolerant-tests/src/test/java/io/trino/faulttolerant/TestFaultTolerantExecutionDynamicFiltering.java Updated config properties from small to default names
plugin/trino-memory/src/test/java/io/trino/plugin/memory/TestMemoryConnectorTest.java Consolidated small/large configs into single set, removed withLargeDynamicFilters helper method, updated test expectations to reflect large filters always enabled
docs/src/main/sphinx/admin/dynamic-filtering.md Updated documentation to remove references to small/large filter distinction and enable-large-dynamic-filters property

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@raunaqmorarka raunaqmorarka force-pushed the raunaq/clean-df-configs branch from dfe2b8a to ce8ea0c Compare December 12, 2025 18:26
Large dynamic filters have been the default for a while
The configs around small dynamic filter limits can be removed now
Since distinction between small and large configs has been
removed, these configs are renamed to drop the reference to large
@raunaqmorarka raunaqmorarka force-pushed the raunaq/clean-df-configs branch from ce8ea0c to 937fece Compare December 15, 2025 04:54
Copy link
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes a lot of sense, thanks

@raunaqmorarka raunaqmorarka merged commit f45239e into trinodb:master Dec 15, 2025
100 checks passed
@raunaqmorarka raunaqmorarka deleted the raunaq/clean-df-configs branch December 15, 2025 15:18
@github-actions github-actions bot added this to the 480 milestone Dec 15, 2025
return smallPartitionedMaxDistinctValuesPerDriver;
}

@Config("dynamic-filtering.small-partitioned.max-distinct-values-per-driver")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't there be a deprecation period during which config still works but using it triggers a warning?

cc @kokosing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not when we're outright removing some functionality.
The "small" configs became irrelevant when large DFs were enabled by default more than an year back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

3 participants