Skip to content

Remove enforce_batch_size_in_joins in symmetric hash join, and deprecate the config #24812

Description

@2010YOUY01

Is your feature request related to a problem or challenge?

Background: - #12969

No this config is not used in most other operators, and SHJ is the only place using it. Outputting incrementally seems to be always better (the behavior when enforce_batch_size_in_joins set to true), so we can only keep this behavior and deprecate the config.

Describe the solution you'd like

  1. Change SHJ implementation to always output incrementally
  2. Deprecate the enforce_batch_size_in_joins config (https://datafusion.apache.org/contributor-guide/api-health.html#deprecation-guidelines)

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions