Skip to content

refactor(ci): enforce the Java 17 SDK build contract - #787

Merged
imbajin merged 187 commits into
apache:masterfrom
hugegraph:fix/java17-sdk-build-contract
Oct 8, 2026
Merged

imbajin merged 187 commits into
apache:masterfrom
hugegraph:fix/java17-sdk-build-contract

Conversation

@imbajin

@imbajin imbajin commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Before → after: candidate SDK setup now rejects unsupported JDKs, installs resolved Maven parent coordinates, and validates a fixed ASF source baseline. Server 1.7 compatibility tests use the official 1.7.0 binary release with its official checksum and cache provenance, instead of rebuilding an old tag. Download failures do not fall back to a source build; existing configuration, HTTP/HTTPS, authentication and consumer tests remain in place.

Reproducible Java 17 CI with fixed ASF SDK and released/candidate Server baselines

Core versions: Java 17, TinkerPop 3.8.1, and Hadoop 3.3.6. The unpublished candidate SDK continues to be built from a fixed ASF commit. A separate upstream compatibility workflow checks current ASF master without changing the reproducible baseline.

Third-party Maven caching reuses the existing Java setup cache while excluding HugeGraph SDK artifacts and manifests. Hubble’s duplicate Maven cache is removed. Build instructions and Docker commands use the repository root context. Hubble excludes the unused org.apdplat:word runtime dependency; configuration remains available and segmentation executes in Server.

HDFS uses the pinned official image. Coverage uploads stay in existing test jobs through OIDC; upload failures and coverage thresholds remain advisory. The required/advisory separation from #791 is preserved.

Shared SDK/archive/manifest production and expanded final-image import/query or Compose verification are tracked in #792 and are outside this PR. Spark and Flink runtime changes remain in #785 and #786.

Validation: prior-head CI passed candidate and release consumer tests, product images, and upstream compatibility. The latest dependency exclusion and official-release fixture changes are pushed for CI validation; no local runtime tests were run.

imbajin added 30 commits October 3, 2026 23:53
- manage compiler, surefire and coverage plugin versions
- keep Java 8 output and use compatible Mockito 4
- resolve optional server repositories, refs and pins
- verify checkout and package servers without install
- share authenticated startup and filter AppleDouble files
- connect repository and fetch refs to all server installers
- reuse the shared installer for Tools and Spark validation
- preserve Loader source caching with verified checkouts
- isolate fixture builds and retain separate HTTPS RPC ports
- preserve Mockito 2 interaction assertions with the equivalent API
- align test-only Byte Buddy dependencies with Mockito 4
- retain the Spring Boot 2 and Java 11 shared baseline
- run shared server selection and installer tests in one CI job
- document Maven 3.6.3 as the minimum source build version
- report missing or ambiguous server archives before exiting
- trigger push checks for client installer changes
- trigger pull request checks for the same source path
- cover the installer exercised by the existing contracts
- encode legacy graph configuration as properties
- keep task and target tests aligned with server contracts
- inherit shared compiler and coverage plugin versions
- preserve Java 8 APIs in compatibility tests
- align Spark 3.5.8 and Scala 2.12.18 with Java 8 bytecode
- replace Scala build plugin and match SLF4J 2 binding
- supply embedded Spark module opens and bounded tests
- isolate configurable server targets and verify Unicode
- run connector CI on Java 11 and 17
- update the shared inventory from the published full reactor runtime
- retain complete bundled license texts and required dependency notices
- cover Jackson suffixed legal entries and the protobuf release license
- keep the Spark 3.5 SLF4J provider in provided scope
- retain the exclusion of the inherited SLF4J 1 binding
- document exact Guava acquisition and driver/executor paths
- state the application classpath override and runtime boundary
- Inherit coverage tooling and keep scoped ORC access on Java 17.
- Update test-only container support and include existing unit coverage.
- Close both distinct clients and cover shared, separate and failing closes.
- Preserve the current Java baseline; shared-base runtime validation is pending.
- remove the provided logging bridge from the runtime inventory
- retain the exact published baseline Log4j API notice
- verify the reactor copy and provider-free assembly legal coverage
- Apply the ORC module opening only on JDK 17 and later.
- Keep default file-test selection when the JDK profile is active.
- Preserve the primary close exception and suppress secondary failures.
- Validate JVM option boundaries and unit/file suites on JDK 11 and 17.
- read plain backup streams without ZIP decoding
- close invalid compressed streams before reporting errors
- cover ZIP and plain backup restore round trips
- allow a dedicated test server URL
- mark the validated submit example as client deploy mode
- explain the remote driver Guava path in cluster deploy mode
- distinguish driver provisioning from executor JAR copies
- Restore profile test selection without global file-test defaults.
- Keep scoped ORC access and late coverage on JDK 11/17 test JVMs.
- Mark client cleanup closed even when either client throws.
- Cover primary, secondary and dual failures followed by retry.
- pass the configured URL to auth backup commands
- pass the same URL to auth restore commands
- keep command and client requests on one test fixture
Upgrade the backend to Spring Boot 3 and Jakarta APIs.
Use a fresh H2 database with complete schema initialization.
Align JAXB and Avatica runtime dependencies.
Preserve runtime data while packaging archives and images.
Carry the verified graph context and Java 17 documentation.
Run Hubble CI on Java 17 with Server 1.7 on Java 11.
Use the shared pinned source checkout and compiler tooling.
Keep server packaging separate from SDK dependency installation.
Verify archive preservation before the distribution gate.
- update license selections for the Hubble Java 17 runtime
- preserve required notices and complete bundled license texts
- regenerate the shared published dependency inventory
Record the selected Server JDK before configuring Hubble Java 17.
Use its explicit home for released Server build and startup.
Keep Hubble package and acceptance on the module JVM.
- retain original module dependencies in the shared inventory
- match the complete reactor dependency-copy result
- verify existing license rows and published artifact hashes
- restore the complete notice for the retained published Jetty dependency
- identify the exact artifact and original JAR entry in the main NOTICE
- preserve every existing distribution notice unchanged
- distinguish the original gRPC runtime identity from Hubble additions
- retain its existing Apache license selection and complete terms
- align inventory classification with the full reactor classpaths
Bind Hikari settings before validating the effective connection.
Check existing H2 metadata read-only before any schema initialization.
Record the schema version only after fresh initialization succeeds.
Verify old database bytes and rows remain unchanged on rejected startup.
- retain three complete bundled Jackson Core license resources
- link the runtime inventory to the supplemental license texts
- verify prefixed and suffixed legal entries against artifact hashes
Accept embedded file URLs with the optional file prefix omitted.
Retain the native cipher when probing encrypted metadata read-only.
Reject INIT and escaped setting names before opening a connection.
Verify shorthand and AES restart plus unchanged encrypted legacy files.
Build the dependency license inventory on Java 17.
Run CodeQL Java reactor compilation on Java 17.
Preserve module target versions and fixture JVM settings.
- include the current tools and loader branches
- preserve the independent shared and client stack
- keep module runtime and source scopes unchanged
- include the current spark connector branch
- retain the verified combined license inventory
- provide the true multi-module cutover baseline

# Conflicts:
#	hugegraph-dist/release-docs/LICENSE
#	hugegraph-dist/release-docs/NOTICE
#	hugegraph-dist/scripts/dependency/known-dependencies.txt
- prepare a Java 11 environment for the published server fixture
- isolate server JAVA_HOME and PATH from the test JVM matrix
- keep Java 11 and 17 CI results independent with fail-fast disabled
- separate the fixed baseline from current master testing
- document one resolved source identity per compatibility run
- reuse existing core tests without duplicate engine matrices
- select the immutable ASF baseline with six characters
- resolve the full identity before fetching Server
- remove the moving master fetch override
- warn instead of failing when Codecov upload errors
- keep upload setup failures outside test job results
- reuse the existing upload steps and authentication
Move Codecov configuration to the repository root.
Keep project and patch coverage statuses informational.
Preserve the client example exclusion relative to the root.
MrJs133
MrJs133 previously approved these changes Oct 7, 2026
- retain upstream advisory CI and independent scans
- preserve the locked SDK identity and reactor closure
- merge isolated parent metadata regression coverage
- reconcile HDFS cleanup and CI documentation
- retain the merged CI policy wording
- avoid early paragraph wrapping
- preserve tables and code blocks

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The move from full 40-character SHAs to six-character locks makes every Toolchain CI run and image build depend on GitHub keeping that prefix unique, and makes Docker and Maven packaging call api.github.com without a token. Keeping the full SHA in the lock values and shortening only log output avoids both; the exact-JDK-17 gate, SDK closure with flatten 1.3.0, fixture reuse of the SDK installer and advisory Codecov changes looked correct. The job-wide id-token permission is already raised in an existing thread and is not repeated here. Evidence: gh api repos/apache/hugegraph/commits/d9abc returns 422 No commit found for SHA while d9abcd and d9abcd4 resolve, so an ambiguous prefix fails rather than picks one; gh api repos/apache/hugegraph/commits/3900381 resolves the head of closed fork PR apache/hugegraph#3274 (hugegraph/hugegraph), so fork PR commits share the prefix namespace; source_commit() in verify_candidate_image_sdk.py calls urllib when CANDIDATE_SOURCE_COMMIT is unset, which is the case in both Dockerfiles and in the antrun verifier executions of hugegraph-dist, loader, tools and hubble-dist poms.

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/scripts/verify_candidate_image_sdk.py Outdated
- pin CI, SDK bootstrap and image builds to full source identities
- remove abbreviated commit API expansion from checkout and packaging
- retain verified full commit context for upstream canary validation
- show short identities in docs and read the execution lock locally
- retain archive member preflight validation
- apply the native data filter during extraction
- cover sequential links and valid internal links
- use the merged ASF request encoding fix
- align workflow and image source identities
- retain full machine locks and short documentation
- pin the ASF binary release URL and official SHA-512
- verify release identity and both archive checksums on cache reuse
- preserve Java 17 candidate builds and existing service startup
- update offline fixture contracts for release downloads and failures
- fix the official 1.7.0 source repository and commit
- reject mismatched callers before cache reads or downloads
- validate and record release source identity in manifests
- cover false source claims in offline fixture contracts
- reuse third-party Maven inputs without HugeGraph SDKs
- remove the duplicate Hubble Maven cache
- correct candidate build and Docker context instructions
keep Server configuration and Struct dependencies

leave segmentation execution in the Server process

avoid bundling the unused GPL Word dependency
- download Server 1.7 from its current ASF location
- retain the official checksum and source validation
- update the fixture identity expectation
- find the unique runnable Server in official packages
- preserve the two-instance configuration and tests
- remove macOS sidecars before graph discovery

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The earlier lock and packaging problems are fixed at this head. Full 40-character SHAs are used throughout, the verifier works offline from its local COMMIT, and Codecov uploads are advisory. The official 1.7.0 release fixture, the exact JDK 17 gate, the SDK module closure and the Hubble archive filter look correct. Two small notes remain: the release download has a single URL, and a short-SHA branch in the SDK installer can no longer be reached. The job-wide id-token: write thread from the earlier head still applies and is not repeated here. Evidence: downloads.apache.org/hugegraph/1.7.0/apache-hugegraph-incubating-1.7.0.tar.gz.sha512 matches the pinned SHA-512, and the same archive is served at archive.apache.org/dist/hugegraph/1.7.0/ (HTTP 200). apache/hugegraph master is identical to e62c961. Locally, runtime_test.py (13 tests), resolve.test.cjs (6), test_candidate_sdk.py (11, one skipped without mvn) and test_run_live_hubble_smoke.py (9) pass on Python 3.12 and Node 25.

Comment thread .github/actions/setup-hugegraph-server/release.py
Comment thread hugegraph-client/assembly/travis/install-candidate-sdk.sh Outdated
- fall back only to the verified official archive
- keep canonical cache identity and checksum failures
- remove the unreachable short-ref rewrite
- detect tarfile extraction filter support before runtime setup
- keep safe extraction mandatory without an unfiltered fallback
- cover unsupported Python with existing archive unit tests
- remove restored Maven negative-cache markers
- retain downloaded third-party dependencies
- recheck repositories after transient failures
@imbajin imbajin changed the title fix(ci): enforce the Java 17 SDK build contract refactor(ci): enforce the Java 17 SDK build contract Oct 8, 2026
@imbajin
imbajin merged commit cfed323 into apache:master Oct 8, 2026
28 checks passed
@imbajin
imbajin deleted the fix/java17-sdk-build-contract branch October 8, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client hugegraph-client hubble hugegraph-hubble loader hugegraph-loader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants