Add IoTDB Edge distribution: ConfigNode + DataNode merged in one JVM - #18538
Merged
Conversation
Produce apache-iotdb-edge-${version}-all-bin from the default distribution
build, targeting resource-constrained edge machines that share the host
with other processes (about 512 MB total process RSS by default).
- EdgeNode (iotdb-core/confignode, org.apache.iotdb.edge): bootstraps the
ConfigNode on a background thread, waits for its internal RPC port, then
starts the DataNode in the same JVM, saving a whole JVM's fixed overhead
(~140 MB validated on x86 and Raspberry Pi 4B).
- One pair of lifecycle scripts only: sbin/start-edge.sh / stop-edge.sh
(plus Windows bats); no separate ConfigNode/DataNode scripts in this
package. start-edge.sh sets CONFIGNODE_HOME explicitly so the ConfigNode
resolves its directories against the installation dir.
- conf/edge-env.sh: fixed small JVM budget (Xmx 224M, direct 96M,
metaspace 160m, SerialGC, Xss320k, ActiveProcessorCount=2).
- Edge-tuned iotdb-system.properties defaults: small thread pools, 1M WAL
buffer, 4M tsfile write buffer, 128M compaction target, CUSTOM region
policy with 1 region group per database, series_slot_num=1, metrics off.
- Validated at 1 Hz mixed read/write on x86 (>=20k series) and
Raspberry Pi 4B (>=10k series, 30-min soak per shape, zero failed points).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18538 +/- ##
=========================================
Coverage 42.23% 42.23%
Complexity 413 413
=========================================
Files 5409 5410 +1
Lines 389925 389995 +70
Branches 50970 50981 +11
=========================================
+ Hits 164685 164728 +43
- Misses 225240 225267 +27 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a resource-constrained IoTDB Edge distribution that runs ConfigNode and DataNode in one JVM.
Changes:
- Adds the merged Edge entry point, launchers, and tuned configuration.
- Adds an isolated
edge-bindistribution and documentation. - Adds Linux integration testing and CI coverage.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/sbin/windows/stop-edge.bat |
Stops the Windows Edge process. |
scripts/sbin/windows/start-edge.bat |
Starts Edge on Windows. |
scripts/sbin/stop-edge.sh |
Safely stops a matching Unix Edge process. |
scripts/sbin/start-edge.sh |
Starts Edge with shared configuration and port checks. |
scripts/conf/windows/edge-env.bat |
Defines Windows Edge JVM limits. |
scripts/conf/edge-env.sh |
Defines Unix Edge JVM limits. |
README.md |
Documents Edge packaging and lifecycle. |
README_ZH.md |
Adds Chinese Edge documentation. |
iotdb-core/node-commons/src/assembly/resources/conf/edge/iotdb-system.properties |
Provides Edge-tuned defaults. |
iotdb-core/confignode/src/main/java/org/apache/iotdb/edge/EdgeNode.java |
Bootstraps both nodes in one JVM. |
iotdb-core/confignode/src/main/i18n/zh/org/apache/iotdb/confignode/i18n/ConfigNodeMessages.java |
Adds Chinese Edge messages. |
iotdb-core/confignode/src/main/i18n/en/org/apache/iotdb/confignode/i18n/ConfigNodeMessages.java |
Adds English Edge messages. |
integration-test/src/test/java/org/apache/iotdb/edge/it/IoTDBEdgeBasicIT.java |
Tests packaged Edge startup and read/write behavior. |
integration-test/src/main/java/org/apache/iotdb/itbase/category/EdgeIT.java |
Defines the Edge test category. |
integration-test/src/assembly/mpp-test.xml |
Excludes Edge-only configuration. |
integration-test/pom.xml |
Adds the Edge integration-test profile. |
distribution/src/assembly/resources/conf-edge/logback-edge.xml |
Defines merged-process logging. |
distribution/src/assembly/edge.xml |
Assembles the Edge distribution. |
distribution/src/assembly/datanode.xml |
Excludes Edge resources from DataNode packages. |
distribution/src/assembly/confignode.xml |
Excludes Edge resources from ConfigNode packages. |
distribution/src/assembly/all.xml |
Keeps Edge files out of the regular all-bin. |
distribution/pom.xml |
Builds and checksums the Edge artifact. |
CLAUDE.md |
Documents Edge architecture and validation. |
.github/workflows/edge-it.yml |
Builds and tests Edge in CI. |
Suppressed comments (1)
distribution/src/assembly/resources/conf-edge/logback-edge.xml:241
PIPEis also never defined as an appender, although all three pipe logger namespaces reference it. This leaves Logback with unresolved references and no dedicated pipe log. Add onePIPEappender for these references or remove the references if root-only logging is intended.
<logger level="info" name="org.apache.iotdb.pipe.api">
<appender-ref ref="PIPE"/>
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
HTHou
approved these changes
Aug 31, 2026
2 tasks
This was referenced Aug 31, 2026
JackieTien97
added a commit
that referenced
this pull request
Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
IoTDB's default configuration and startup parameters assume the machine is exclusively dedicated to IoTDB (heap sized from machine memory: DataNode 50% uncapped, ConfigNode 30% capped at 8G). On edge machines that also run other processes this is far too aggressive. This PR adds an IoTDB Edge distribution —
apache-iotdb-${version}-edge-bin— whose defaults target a total process RSS of ~512 MB, validated on Raspberry Pi 4B class devices.Changes
New entry point (one JVM, both nodes)
org.apache.iotdb.edge.EdgeNode(iniotdb-core/confignode, which already depends oniotdb-server): bootstraps the ConfigNode on a background thread, waits for its configured internal RPC address and port to accept connections, then starts the DataNode in the same JVM. This saves a whole JVM's fixed overhead (metaspace, code cache, GC structures, thread stacks) — ~140 MB measured. All logs go through i18n'dConfigNodeMessagesconstants (en/zh in key parity).MetricServicesingleton is safe because both node metric levels default toOFFin the edge config.One pair of lifecycle scripts only
sbin/start-edge.sh/stop-edge.sh(+ Windowsstart-edge.bat/stop-edge.bat). No separate ConfigNode/DataNode scripts in this package.start-edge.shsetsCONFIGNODE_HOMEexplicitly so the ConfigNode resolves its directories against the installation dir rather than the process working directory (a real pitfall found during testing).conf/edge-env.sh(+.bat): fixed small JVM budget, overridable via environment variables —Xmx 224M/Xms 64M, direct 96M, metaspace 160m, SerialGC,Xss320k,ActiveProcessorCount=2.Edge-tuned defaults (
conf/iotdb-system.propertiesin the edge package only)schema/data_region_group_extension_policy=CUSTOMwith 1 region group per database,series_slot_num=1.cn/dn_metric_level=OFF.model_inference_execution_thread_counthas been removed. The pre-existing shared property-loading behavior is unchanged.QUERY_FREQUENCY/PIPEappender references have been removed from the Edge logback configuration.Packaging
distribution/src/assembly/edge.xml, Edge system defaults underiotdb-core/node-commons/src/assembly/resources/conf/edge/, and the merged logback configuration underdistribution/src/assembly/resources/conf-edge/; the defaultmvn package -pl distribution -ambuild now also producesapache-iotdb-${version}-edge-bin(dir/zip + sha512), alongside the unchanged regular packages.Validation
Full test report (machine specs incl. Raspberry Pi 4B): https://timechor.feishu.cn/docx/Hlf0dyQ4FoQsf4xEFnSc4TTknIg?from=from_copylink
mvn clean package -pl distribution -am -DskipTestspasses; full-reactormvn test-compile -DskipTestsalso passes for both the default andwith-zh-localeprofiles (50 modules each)..github/workflows/edge-it.ymlcoverage: the packaged Edge startup/shutdown lifecycle, Tree and Table read/write ITs, packaged configuration checks, and a non-default ConfigNode address (127.0.0.2on Linux).