Skip to content

Add IoTDB Edge distribution: ConfigNode + DataNode merged in one JVM - #18538

Merged
JackieTien97 merged 4 commits into
masterfrom
edge-version
Aug 31, 2026
Merged

Add IoTDB Edge distribution: ConfigNode + DataNode merged in one JVM#18538
JackieTien97 merged 4 commits into
masterfrom
edge-version

Conversation

@JackieTien97

@JackieTien97 JackieTien97 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Motivation

IoTDB's default configuration and startup parameters assume the machine is exclusively dedicated to IoTDB (heap sized from machine memory: DataNode 50% uncapped, ConfigNode 30% capped at 8G). On edge machines that also run other processes this is far too aggressive. This PR adds an IoTDB Edge distributionapache-iotdb-${version}-edge-bin — whose defaults target a total process RSS of ~512 MB, validated on Raspberry Pi 4B class devices.

Changes

New entry point (one JVM, both nodes)

  • org.apache.iotdb.edge.EdgeNode (in iotdb-core/confignode, which already depends on iotdb-server): bootstraps the ConfigNode on a background thread, waits for its configured internal RPC address and port to accept connections, then starts the DataNode in the same JVM. This saves a whole JVM's fixed overhead (metaspace, code cache, GC structures, thread stacks) — ~140 MB measured. All logs go through i18n'd ConfigNodeMessages constants (en/zh in key parity).
  • Shared-singleton audit: ports / data dirs / JMX bean names don't collide; the internal MetricService singleton is safe because both node metric levels default to OFF in the edge config.

One pair of lifecycle scripts only

  • sbin/start-edge.sh / stop-edge.sh (+ Windows start-edge.bat / stop-edge.bat). No separate ConfigNode/DataNode scripts in this package. start-edge.sh sets CONFIGNODE_HOME explicitly so the ConfigNode resolves its directories against the installation dir rather than the process working directory (a real pitfall found during testing).
  • Unix and Windows launchers check all seven configured ConfigNode/DataNode service ports before launching the merged JVM. The Windows environment script also loads the Maven-filtered TsFile locale option.
  • conf/edge-env.sh (+ .bat): fixed small JVM budget, overridable via environment variables — Xmx 224M / Xms 64M, direct 96M, metaspace 160m, SerialGC, Xss320k, ActiveProcessorCount=2.

Edge-tuned defaults (conf/iotdb-system.properties in the edge package only)

  • Small fixed thread pools (query 2, compaction 2/1/1, flush 2, MPP exchange 2/2, …) instead of CPU-core-based sizing.
  • 1M WAL buffer, 4M tsfile write buffer, 128M compaction target file size.
  • schema/data_region_group_extension_policy=CUSTOM with 1 region group per database, series_slot_num=1.
  • Compaction throughput capped at 8 MB/s to stay polite on shared disks; cn/dn_metric_level=OFF.
  • The ineffective Edge override for model_inference_execution_thread_count has been removed. The pre-existing shared property-loading behavior is unchanged.
  • Query-frequency and Pipe logs use the root appenders; undefined QUERY_FREQUENCY/PIPE appender references have been removed from the Edge logback configuration.

Packaging

  • distribution/src/assembly/edge.xml, Edge system defaults under iotdb-core/node-commons/src/assembly/resources/conf/edge/, and the merged logback configuration under distribution/src/assembly/resources/conf-edge/; the default mvn package -pl distribution -am build now also produces apache-iotdb-${version}-edge-bin (dir/zip + sha512), alongside the unchanged regular packages.
  • Existing assemblies exclude Edge-only configuration and scripts, including the Windows preflight helper. After the review fixes, all five existing distribution zips retain the same file lists, modes, and non-JAR contents as before those fixes; rebuilt JARs are not claimed to be byte-identical.

Validation

Full test report (machine specs incl. Raspberry Pi 4B): https://timechor.feishu.cn/docx/Hlf0dyQ4FoQsf4xEFnSc4TTknIg?from=from_copylink

  • Raspberry Pi 4B (4×Cortex-A72/8G/SD): ≥ 10,000 series across tree-aligned / tree-non-aligned / table models, 30-min soak per shape, zero failed points, zero OOM; idle RSS ~194 MB.
  • Edge package smoke on Pi: start-edge → CLI insert/select → stop-edge, clean shutdown verified.
  • mvn clean package -pl distribution -am -DskipTests passes; full-reactor mvn test-compile -DskipTests also passes for both the default and with-zh-locale profiles (50 modules each).
  • The packaged Edge IT passes all three tests locally with both the default ConfigNode address and a non-loopback local address. The PowerShell preflight helper also passes 14 local port/configuration cases.
  • Dedicated .github/workflows/edge-it.yml coverage: the packaged Edge startup/shutdown lifecycle, Tree and Table read/write ITs, packaged configuration checks, and a non-default ConfigNode address (127.0.0.2 on Linux).
  • A separate Windows job in that workflow covers filtered default/Chinese locale options, custom configuration and paths with spaces, all seven occupied ports, omitted/default properties, comments/duplicates, and invalid port values.

Produce apache-iotdb-edge-${version}-all-bin from the default distribution
build, targeting resource-constrained edge machines that share the host
with other processes (about 512 MB total process RSS by default).

- EdgeNode (iotdb-core/confignode, org.apache.iotdb.edge): bootstraps the
  ConfigNode on a background thread, waits for its internal RPC port, then
  starts the DataNode in the same JVM, saving a whole JVM's fixed overhead
  (~140 MB validated on x86 and Raspberry Pi 4B).
- One pair of lifecycle scripts only: sbin/start-edge.sh / stop-edge.sh
  (plus Windows bats); no separate ConfigNode/DataNode scripts in this
  package. start-edge.sh sets CONFIGNODE_HOME explicitly so the ConfigNode
  resolves its directories against the installation dir.
- conf/edge-env.sh: fixed small JVM budget (Xmx 224M, direct 96M,
  metaspace 160m, SerialGC, Xss320k, ActiveProcessorCount=2).
- Edge-tuned iotdb-system.properties defaults: small thread pools, 1M WAL
  buffer, 4M tsfile write buffer, 128M compaction target, CUSTOM region
  policy with 1 region group per database, series_slot_num=1, metrics off.
- Validated at 1 Hz mixed read/write on x86 (>=20k series) and
  Raspberry Pi 4B (>=10k series, 30-min soak per shape, zero failed points).
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.23%. Comparing base (6aec4b8) to head (613de3b).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
.../src/main/java/org/apache/iotdb/edge/EdgeNode.java 0.00% 37 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master   #18538   +/-   ##
=========================================
  Coverage     42.23%   42.23%           
  Complexity      413      413           
=========================================
  Files          5409     5410    +1     
  Lines        389925   389995   +70     
  Branches      50970    50981   +11     
=========================================
+ Hits         164685   164728   +43     
- Misses       225240   225267   +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@HTHou
HTHou requested a balanced review from Copilot August 31, 2026 01:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a resource-constrained IoTDB Edge distribution that runs ConfigNode and DataNode in one JVM.

Changes:

  • Adds the merged Edge entry point, launchers, and tuned configuration.
  • Adds an isolated edge-bin distribution and documentation.
  • Adds Linux integration testing and CI coverage.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
scripts/sbin/windows/stop-edge.bat Stops the Windows Edge process.
scripts/sbin/windows/start-edge.bat Starts Edge on Windows.
scripts/sbin/stop-edge.sh Safely stops a matching Unix Edge process.
scripts/sbin/start-edge.sh Starts Edge with shared configuration and port checks.
scripts/conf/windows/edge-env.bat Defines Windows Edge JVM limits.
scripts/conf/edge-env.sh Defines Unix Edge JVM limits.
README.md Documents Edge packaging and lifecycle.
README_ZH.md Adds Chinese Edge documentation.
iotdb-core/node-commons/src/assembly/resources/conf/edge/iotdb-system.properties Provides Edge-tuned defaults.
iotdb-core/confignode/src/main/java/org/apache/iotdb/edge/EdgeNode.java Bootstraps both nodes in one JVM.
iotdb-core/confignode/src/main/i18n/zh/org/apache/iotdb/confignode/i18n/ConfigNodeMessages.java Adds Chinese Edge messages.
iotdb-core/confignode/src/main/i18n/en/org/apache/iotdb/confignode/i18n/ConfigNodeMessages.java Adds English Edge messages.
integration-test/src/test/java/org/apache/iotdb/edge/it/IoTDBEdgeBasicIT.java Tests packaged Edge startup and read/write behavior.
integration-test/src/main/java/org/apache/iotdb/itbase/category/EdgeIT.java Defines the Edge test category.
integration-test/src/assembly/mpp-test.xml Excludes Edge-only configuration.
integration-test/pom.xml Adds the Edge integration-test profile.
distribution/src/assembly/resources/conf-edge/logback-edge.xml Defines merged-process logging.
distribution/src/assembly/edge.xml Assembles the Edge distribution.
distribution/src/assembly/datanode.xml Excludes Edge resources from DataNode packages.
distribution/src/assembly/confignode.xml Excludes Edge resources from ConfigNode packages.
distribution/src/assembly/all.xml Keeps Edge files out of the regular all-bin.
distribution/pom.xml Builds and checksums the Edge artifact.
CLAUDE.md Documents Edge architecture and validation.
.github/workflows/edge-it.yml Builds and tests Edge in CI.
Suppressed comments (1)

distribution/src/assembly/resources/conf-edge/logback-edge.xml:241

  • PIPE is also never defined as an appender, although all three pipe logger namespaces reference it. This leaves Logback with unresolved references and no dedicated pipe log. Add one PIPE appender for these references or remove the references if root-only logging is intended.
    <logger level="info" name="org.apache.iotdb.pipe.api">
        <appender-ref ref="PIPE"/>

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread iotdb-core/confignode/src/main/java/org/apache/iotdb/edge/EdgeNode.java Outdated
Comment thread iotdb-core/node-commons/src/assembly/resources/conf/edge/iotdb-system.properties Outdated
Comment thread scripts/conf/windows/edge-env.bat
Comment thread distribution/src/assembly/resources/conf-edge/logback-edge.xml Outdated
Comment thread scripts/sbin/windows/start-edge.bat
Comment thread distribution/src/assembly/edge.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants