Skip to content

fix(store): complete RPC and scan lifecycle - #3303

Merged
imbajin merged 11 commits into
apache:masterfrom
hugegraph:fix/store-scan-resource-lifecycle
Oct 10, 2026
Merged

imbajin merged 11 commits into
apache:masterfrom
hugegraph:fix/store-scan-resource-lifecycle

Conversation

@imbajin

@imbajin imbajin commented Oct 9, 2026 •

Copy link
Copy Markdown
Member
image Client and server distinguish half-close from cancellation, retain final batches, and share per-call worker/iterator ownership through cleanup. Unary query/count use the same ownership path; count keeps partition work parallel.

Scan completion rechecks producer completion and the queue after an empty timed poll. Cancellation signals the currently owned reader, including one-shot callers, without leaking its interrupt into a subsequent scan. Native reads still retain ownership until they return. Expired batch-stream cleanup runs outside the admission monitor. Aggregate shutdown reports UNAVAILABLE instead of successful partial results.

Global admission and terminal callbacks drain before native shutdown. The existing gRPC executor uses an unbounded callback queue to avoid dropping terminal callbacks; core/max and backlog tradeoffs are documented. Deterministic race, half-close, thread-reuse, Netty cancellation and cleanup regressions passed focused local validation and independent review; backend integration results are reported by the PR checks. Scan benchmarking and full process recovery remain outside the current evidence.

Long-term RPC, recovery, runtime upgrade and provider contracts are consolidated in docs/storage-lifecycle.md. Merge this prerequisite before the incremental Native shutdown PR.

image

Paired website documentation: hugegraph/hugegraph-doc#39 and apache/hugegraph-doc#513. Coordinate configuration documentation and code merges.

- align client cancellation and server half-close behavior
- publish final batches outside response parsing locks
- drain RPC callbacks and scan owners before storage close
- retain TTL cleanup failures and finish interrupted Raft joins
- preserve executable packaging and lifecycle CI coverage
- retain parser failures across completion races
- drain heartbeat producers despite interruption
- restore interrupts after native database teardown
- cover real native close in an isolated subprocess
- keep client and server request lifecycle changes together
- retain scoped scan and query cleanup before database close
- partition lifecycle tests without changing their assertions
- defer global RPC and TTL shutdown to a dependent PR
@imbajin imbajin changed the title fix(store): drain scan and native resources fix(store): preserve RPC and scan lifecycles Oct 9, 2026
- close global RPC admission before request cleanup waits
- retain batch half-close credit and explicit client cancellation
- track unary query and parallel count resource owners
- preserve failed native cleanup independently of RPC completion
- keep matching regressions in the RPC prerequisite
@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.13514% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.71%. Comparing base (d00efce) to head (dd0b560).

Files with missing lines Patch % Lines
...aph/store/client/query/CommonKvStreamObserver.java 82.60% 3 Missing and 9 partials ⚠️
...java/org/apache/hugegraph/store/HgStoreEngine.java 75.00% 3 Missing and 3 partials ⚠️
...a/org/apache/hugegraph/store/HeartbeatService.java 83.33% 3 Missing ⚠️
...va/org/apache/hugegraph/store/PartitionEngine.java 92.85% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3303      +/-   ##
============================================
+ Coverage     41.42%   41.71%   +0.28%     
- Complexity     6931     7004      +73     
============================================
  Files           762      762              
  Lines         66773    66838      +65     
  Branches       8939     8956      +17     
============================================
+ Hits          27662    27880     +218     
+ Misses        35957    35770     -187     
- Partials       3154     3188      +34     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- retain off-thread dispatch during executor saturation
- verify cancellation and drain through real Netty RPCs
- document callback queue and shutdown boundaries
- interrupt active readers before iterator cleanup waits
- verify context cancellation and deadline cleanup
- propagate half-close worker failures into assertions
@imbajin imbajin changed the title fix(store): preserve RPC and scan lifecycles fix(store): complete RPC and scan lifecycle Oct 9, 2026
- listen to context cancellation during blocked unary work
- remove listeners while retaining iterator cleanup accounting
- verify query and count cancellation through real Netty RPCs
- interrupt owned batch readers before iterator locks
- keep closed partition iterators terminal
- refresh the timeout when accepting a response
- cover blocked shutdown and parsing boundaries
- combine RPC, recovery and provider guidance
- preserve WAL and native ownership safety contracts
- remove PR-stage notes and repeated explanations
- update entry points to the canonical guide
- recheck the queue after an empty timed poll
- preserve producer publication before completion
- cover final-batch delivery at the poll boundary
Interrupt active one-shot callers and preserve thread reuse.
Close expired streams outside the admission monitor.
Report aggregate shutdown cancellation as unavailable.
Cover cancellation, terminal delivery and retained cleanup.
@imbajin
imbajin merged commit 72085aa into apache:master Oct 10, 2026
51 of 53 checks passed
@imbajin
imbajin deleted the fix/store-scan-resource-lifecycle branch October 10, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants