Repository navigation
Conversation
- require confirmed loading mode before cleanup - keep cleanup state local and reset it on clone - cover failed startup and cleanup error paths - document network uncertainty and shared mode limits
- log the graph when a mode write loses its response - preserve the original checked transport exception - retain cleanup ownership and avoid blind mode resets - verify the diagnostic and explain manual recovery
- include the latest mode acquisition guard - preserve current ASF launcher and fixture fixes - keep Hubble changes available for review
- retain loader ownership through cleanup and recovery - preserve typed file progress during cancellation - serialize job admission with metadata deletion - verify transaction rollback and stale callbacks
- align the Hubble cleanup branch with current master - preserve the validated Loader and Hubble source bytes - retain lifecycle and checkpoint regression coverage
- Require graph-instance loading owner capability. - Reuse one owner tuple for uncertain request outcomes. - Release ownership after all loader workers terminate. - Document server compatibility and orphan recovery.
- Lock Loader SDK and runtime to the protocol-1 Server. - Retain all import profiles on the matching candidate. - Verify official Server 1.7 rejects import before mutation. - Record per-consumer source inputs and document coverage.
- route Ingest and graph cleanup through scoped job deletion - preserve permission and remote-drop failure boundaries - retry rejected dispatch after SQL and commit failures - isolate polling writes and retire owners after commit - document and verify deletion and recovery contracts
- propagate Spring transaction and savepoint failures - retain earlier owners when the polling transaction aborts - verify real deadlocks and ordinary SQL failure isolation - document the polling rollback boundary
- Confirm protocol support before every import mode. - Clean RESTORING only after a successful enum change. - Match Hubble and Loader image import fixtures to Server. - Verify legacy refusal while preserving traversal and UI checks.
- Preserve loading ownership and restoring cleanup. - Retain released SDK helpers in source assemblies. - Include the upstream JSON list parser and its coverage.
- inherit merged source release helper migration - retain upstream JSON list support - preserve reviewed Hubble deletion and recovery changes
Merge the published Hubble cleanup dependency normally. Preserve the reviewed loading-owner and CI files. Keep current Apache master and source-package helpers.
github-actions
Bot
added
hubble
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #815 +/- ##
============================================
+ Coverage 47.06% 49.14% +2.07%
- Complexity 4518 4833 +315
============================================
Files 602 603 +1
Lines 29452 30154 +702
Branches 3456 3629 +173
============================================
+ Hits 13862 14819 +957
+ Misses 14308 13942 -366
- Partials 1282 1393 +111 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
- use deletion-specific stopping while keeping stop requests strict - preserve completed states and wait for existing owner cleanup - cover H2 completion races, closure refusal and completion waits - document deletion of tasks that finish after enumeration
- Pin Loader and Hubble imports and Loader image smoke to the paired Server fix. - Keep existing SDK defaults and official Server 1.7 refusal controls. - Verify workflow syntax, source resolution and frozen source/class bindings.
Keep fixed owner handles for acquire and cleanup Preserve legacy imports and RESTORING behavior Remove unrelated Hubble and checkpoint changes
- close task and example loaders after loading - close unused loaders on save, dispatch or transaction failure - remove owned shutdown hooks and preserve unused checkpoints - defer metadata and concurrency recovery to separate work
Limit manual NONE recovery to legacy Servers Explain same-tuple and same-principal REST release Document that repeated shutdown does not retry release
- limit unused-loader cleanup to initial task creation - retain task-exit and built-in example guidance - leave resume and retry failures in the deferred scope
Keep owner tokens out of diagnostics. Document privileged recovery with epoch and revision checks. Exercise legacy Server imports in release CI. Verify owner requests tolerate revision capabilities.
Close loaders when resume or retry dispatch fails. Preserve stopped states when shutdown rethrows the load failure. Report independent cleanup errors without replacing the primary cause. Cover native loader exception and dispatch boundaries.
- retain independent mode reset errors during shutdown - preserve direct fatal cleanup errors and identity - prevent cyclic cleanup exception diagnostics - cover actual loader resource cleanup and reset failures
bitflicker64
left a comment
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Score: 9/10 (10 - 0.5 - 0.5 for two Minor). Summary: the owner tuple stays fixed across acquire and release, the 404 fallback only triggers on the Jersey not-found reply, and cleanup keeps the release failure with the close failure suppressed. The earlier points about the masked release error, the RESTORING prerequisite, the lost-handle recovery and the CI run of LegacyServerCompatibilityTest are addressed at this head. Evidence: read the full diff with LoadContext, HugeGraphLoader, GraphsAPI and ServerException, and LoadingMode/GraphsAPI at apache/hugegraph#3331 head 7762b90. LoadingModeOwnerTest (4 tests) and LoadContextTest (35 tests) pass locally. The release-1.7 loader leg ran LegacyServerCompatibilityTest (2 tests, 0 failures), and all checks pass at 324206d. Open: apache/hugegraph#3331 is not merged yet, so the protocol-1 path has only been checked against stub servers in this build. I did not run the two-Loader scenario against the paired Server.
- direct release failures to administrator recovery - explain that related imports must stop and drain - align diagnostics with the documented cleanup flow
bitflicker64
left a comment
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The only change since 324206d is the release-failure message, which now points at the Server administrator recovery procedure instead of a retry, so that earlier Minor thread is addressed; the owner handle, the narrow 404 fallback and the cleanup ordering in stopThenShutdown() are unchanged and still read correctly. One new Minor note on where the README places the ownership section, inline. Evidence: full diff 8e6ed84..73ad996 and the delta 324206d..73ad996 read with LoadContext, HugeGraphLoader.setGraphMode/load/stopThenShutdown, client GraphsAPI and LoadingModeOwner, and LoadingMode.update and the Server GraphsAPI at apache/hugegraph#3331 head 7762b90; walked a rejected acquire (for example a leftover RESTORING) through shutdown: the release of the unregistered token returns released with the current mode, which verify accepts, so the original rejection is not masked. LoadContextTest still asserts only the unchanged Failed to release LOADING owner prefix. No tests were run for this review. At review time the Module validation client-ci and loader-ci matrix jobs for this head (including the release-1.7 leg that runs LegacyServerCompatibilityTest) had been queued for about three and a half hours, so no CI result covers this head yet. The Hubble release gap (open Minor thread on LoadContext.java line 238) is handled by the paired #814 and is not repeated here.
- move the ownership and recovery section into import usage - link protocol guidance from startup cleanup behavior - document paired Hubble cleanup and shutdown requirements
- preserve the published loader cleanup ancestry - include the current ASF build configuration - prepare the isolated Hubble feedback delivery
- close embedded loaders without saving parsed offsets - wait for callbacks before short metadata deletions - preserve cleanup evidence and document operator recovery - verify pause and failed resume through native loading
- allow metadata deletion after the old runtime has ended - separate Server graph recovery from Hubble metadata deletion - retain current-process cleanup failure responsibility
- preserve task reservations and short delete transactions - retain loader cleanup and immutable progress behavior - add owner protocol and graph binding guidance
- return the canonical missing owner endpoint response - retain legacy reset and native close failure assertions - preserve strict owner protocol fallback behavior
bitflicker64
left a comment
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Score: 9.5/10 (10 - 0.5 for one new Minor). Summary: the only change since 73ad996 is c1b2575, which moves the ownership and recovery section into 3.1 with an anchor from the startup/cleanup subsection and adds the Hubble deployment note; the owner handle, the narrow 404 fallback and the release/close ordering are unchanged and still read correctly. One new Minor inline: line 220 describes enum-mode rejection more narrowly than line 179 and the paired Server. Evidence: read the delta 73ad996..c1b2575 and re-read the full PR diff with LoadContext, HugeGraphLoader.setGraphMode/stopThenShutdown, client GraphsAPI/LoadingModeOwner and the Spark connector HGLoadContext. Checked the wire format against apache/hugegraph#3331 head 33887ab (LoadingMode, GraphsAPI and HugeGraphAuthProxy): the path .../graphs/{graph}/mode/owners, GET capability (protocol, instance, epoch, revision), PUT body (action, instance, epoch, token) and receipt (protocol, instance, epoch, token, state, mode) all match. The README recovery body and confirm string also match. Locally LoadingModeOwnerTest (4 tests) and LoadContextTest (35 tests) pass. CI: the required check-license-header passes; Module validation for c1b2575 is still queued. The 73ad996 run passed every module job, including the release-1.7 loader leg that runs LegacyServerCompatibilityTest (2 tests, 0 failures); its affected-module-tests failed only with StaleInputError because the head moved. Not run: the paired Server and two-Loader scenario, since apache/hugegraph#3331 is still open.
My earlier points:
- describe every enum mode write blocked by active owners - document Spark connector and older Loader compatibility - direct users to stop, drain and retry existing workflows
- identify the checkpoint fixture as a legacy Server - preserve strict ownership endpoint fallback in the SDK - keep native parser and resume regressions reachable
- hold active mock runs until every stop is requested - reuse the existing latch to preserve queued-first ordering - retain cleanup assertions and production behavior
- inherit the shared stop fanout latch in the fixture - preserve SDK ownership and checkpoint behavior - retain the original stacked dependency ancestry
- adopt merged CDC validation and Go dependency selectors - retain the current Hubble cleanup implementation - create a native synchronization event for the updated base
- inherit the verified Hubble base synchronization - retain the existing SDK ownership implementation - preserve the twelve-file SDK layer above its dependency
- use standard suppression with a direct identity guard - remove recursive throwable graph scanning - retain failure identity and native cleanup coverage
- reuse existing uploading directories for later chunks - retain first reservation alias checks and upload quotas - verify real chunks, alias rejection and operation counts
- inherit upload reservation reuse across file chunks - retain standard Loader cleanup failure suppression - preserve the existing SDK ownership projection
- sync the merged ASF CDC fixture helper - retain the verified upload reservation simplification - inherit matching CI selection regression coverage
- inherit the exact ASF master helper integration - retain SDK ownership and failure suppression - preserve the verified upload optimization
3 participants
Purpose of the PR
When two Loader processes import one graph on the same live Server, the first to finish can reset LOADING while the other still imports. Use the paired Server ownership API so an ordinary import releases only its own handle after its workers stop.
Dependency and comparison scope
Depends on #814. The SDK projection changes 12 files above that source. ASF targets master, so its current 28-file comparison also includes #814; recheck the remaining SDK delta after #814 merges.
Hubble deployments require task cleanup #814 and application shutdown #818 together with this SDK change. The SDK alone cannot close Hubble's retained native Loaders. The ownership guarantee becomes available only after Server #3331 is merged and released.
Main Changes
While ownership handles are active, the Server rejects every enum graph-mode PUT for that graph, including NONE, LOADING and RESTORING. Avoid running the Spark connector, older Loaders or restore imports concurrently with owned imports; stop and drain the owned imports, release or recover ownership, then retry through the existing client workflow.
The inherited checkpoint fixture now explicitly returns the recognized legacy missing-endpoint response, keeping its native parser and resume regression tests on the intended insert path.
Verifying these changes
Client/Loader unit suites and the inherited Hubble cleanup/transaction controls pass. The complete #814/#815/#818 stack with the paired Server passes the approved runtime gates: Chrome completes three independent single-CSV jobs, and direct Server reads verify their data and final NONE mode.
The existing same-job legacy path starts nine CSV mappings, exercising eight active imports and one queued task. The original packaged stop script exits naturally in about 4.1 seconds within its default 30-second deadline. Independent post-exit H2, native-resource and Server reads confirm PAUSED task state, completed cleanup and the expected imported data. The UI control does not claim same-job multi-file submission or observed UI overlap; distributed and abrupt-crash scenarios remain outside this validation.
New-head CI remains to be checked after this source update.
Does this PR potentially affect the following parts?
Documentation Status
Doc - TODODoc - Done: Loader README.Doc - No NeedPaired PR: hugegraph#68
Conceptual overview
Protocol-1 imports retain one fixed owner handle and release it after workers stop. Only a recognized missing endpoint uses legacy mode; same-Server ownership and administrator recovery require paired Server #3331.
Conceptual illustration; not test or release-acceptance evidence.