Skip to content

[feature](catalog) support medium allocation mode#63011

Open
Ryan19929 wants to merge 1 commit intoapache:masterfrom
Ryan19929:medium-pr1-table-property
Open

[feature](catalog) support medium allocation mode#63011
Ryan19929 wants to merge 1 commit intoapache:masterfrom
Ryan19929:medium-pr1-table-property

Conversation

@Ryan19929
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #58897

Problem Summary:

This is the first PR of a series that enables RESTORE to choose the target
storage medium in a clear and consistent way.

To support that RESTORE feature, Doris first needs a stable table/partition
metadata model for storage medium allocation. This series also makes storage
medium selection consistent between CREATE TABLE and RESTORE.

The complete series will introduce:

  • STRICT / ADAPTIVE medium allocation semantics.
  • CREATE TABLE hard binding: specifying storage_medium means STRICT; omitting it means ADAPTIVE.
  • RESTORE medium properties, including same_with_upstream, in follow-up PRs.
  • A unified MediumDecisionMaker for restore scenarios in follow-up PRs.
  • FE-BE per-task fallback control (allow_medium_fallback) in follow-up PRs.

This PR only builds the table-level metadata foundation. It does not change RESTORE behavior yet.

Main changes in this PR:

  • Add MediumAllocationMode (STRICT / ADAPTIVE).
  • Persist mediumAllocationMode in DataProperty.
  • Keep TableProperty.mediumAllocationMode as a rebuilt cache from table properties.
  • Map CREATE TABLE storage_medium to STRICT; unspecified medium remains ADAPTIVE.
  • Change SystemInfoService.selectBackendIdsForReplicaCreation and cloud override to accept MediumAllocationMode instead of a boolean.
  • Keep old image compatibility by defaulting missing mediumAllocationMode to ADAPTIVE.

This also fixes the previous storageMediumSpecified problem: the old boolean was not persisted because it had no @SerializedName.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Introduce MediumAllocationMode as the table and partition metadata
foundation for storage medium allocation semantics. CREATE TABLE keeps
the existing user syntax: explicitly specifying storage_medium maps to
STRICT, while omitting it maps to ADAPTIVE.

Persist the mode in DataProperty, rebuild the table-level cache in
TableProperty, and migrate SystemInfoService and its callers from the
old boolean flag to the enum so STRICT placement cannot silently fall
back. Old images that do not contain the new field default to ADAPTIVE,
which matches the previous behavior because storageMediumSpecified was
not persisted.

Add FE unit coverage for enum parsing, DataProperty compatibility,
TableProperty cache behavior, and STRICT/ADAPTIVE backend selection, plus
a regression test covering persistence across FE restart.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Ryan19929
Copy link
Copy Markdown
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants