branch-4.1: [fix](load) correct quorum participants for incremental streams #66016 - #66171
Open
github-actions[bot] wants to merge 1 commit into
Open
branch-4.1: [fix](load) correct quorum participants for incremental streams #66016#66171github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
### What problem does this PR solve?
Problem Summary:
Auto partition loads may create incremental load streams after another
source has already reached write quorum.
In `VTabletWriterV2::_quorum_success()`, a stream that had not entered
the current close stage was implicitly counted as finished — the check
only looked at `unfinished_streams` membership and cancellation.
Incremental streams never enter the first-stage `close_wait` participant
set (`_non_incremental_streams()`), so they contributed free "finished"
votes.
As a result the first-stage `close_wait` could return before every
source had sent CLOSE_LOAD. The early source then sent CLOSE_LOAD on its
incremental streams while the late sources had not opened theirs yet. A
destination's incremental `LoadStream` counts senders dynamically
(`add_source()` bumps `_total_streams`), so it concluded that all
senders had closed and committed the tablet early, which surfaces as
double close and segment number mismatch errors.
This PR excludes streams that have not entered the current close stage
from V2 quorum accounting (`!stream->is_closing()`), matching the V1
participant semantics while preserving the two-stage close algorithm. V1
already has the equivalent guard: `IndexChannel::_quorum_success()`
skips a node channel when `check_status()` fails, and `check_status()`
is `none_of({_cancelled, !_eos_is_produced})`, i.e. it also requires the
channel to have been `mark_close()`d for the current stage.
Also documents why leaving the first-stage `close_wait` on quorum
success still preserves the cross-source fence.
Tests:
-
`TestVTabletWriterV2.quorum_excludes_streams_not_closing_in_current_stage`
- `auto_partition_quorum_race_docker` (20 attempts, 1 FE + 3 BE,
`enable_quorum_success_write=true`)
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
|
run buildall |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picked from #66016