fix: time out idle HTTP connections and harden start.sh - #7003
Open
SeriousCoding789 wants to merge 12 commits into
Open
SeriousCoding789 wants to merge 12 commits into
SeriousCoding789 wants to merge 12 commits into
Conversation
The BlockId overload had no callers. Every caller uses getBranch(Sha256Hash, Sha256Hash), which rejects a branch whose parent is missing.
download() now prints a notice and returns non-zero; release files must be fetched and verified manually. It returns instead of exiting so that restart() still completes when rebuildManifest reaches it.
Connections that send nothing could hold every maxHttpConnectNumber slot until the connector's 30-second idle timeout and lock other clients out. When the limit is reached, the open connections now time out after 10 seconds of inactivity. Connections still being accepted at that moment keep the connector's default idle timeout.
317787106
reviewed
Sep 28, 2026
Downloads: - TLS certificates are verified and a failed request leaves no file. - Release jars must carry a valid GPG signature from the release key listed under "Integrity Check" in README.md. A jar that fails is not used: --upgrade keeps the old jar and --download the existing one. - The mainnet config comes from java-tron and the Nile config from nile-testnet. Bugs: - --stop looped forever, the node was restarted two or three times per run, and -c was passed again on every start. - A jar argument such as my.jar was ignored, and -j with a path lost the path before starting. - --net ignored an existing config file; --net and --release kept going after a failed download. - Linux heap sizing needed bc, and the macOS memory check failed with an expr error. - Manifest rebuilding used a wrong URL and never ran the plugin. It now runs and, by default, rewrites manifests of 128 MB or more. - The latest release was picked in name order, clone failures were ignored, $darwin was never set, and failures exited with status 0. Platforms: - start.sh reads the Java version and architecture of its JVM. JDK 8 keeps the CMS options and JDK 17 uses the ZGC options from start.sh.simple, which is removed. ARM64 downloads the aarch64 jars and skips manifest rebuilding. - -s stops the node and the open file limit is raised to 65535, as in start.sh.simple. The default JVM_MX, used on macOS, is 12g. README and shell.md describe the single script.
SeriousCoding789
changed the title
Downloads: - TLS certificates are verified and a failed request leaves no file. - Release jars must carry a valid GPG signature from the release key listed under "Integrity Check" in README.md. A jar that fails is not used: --upgrade keeps the old jar and --download the existing one. - The mainnet config comes from java-tron and the Nile config from nile-testnet. Bugs: - --stop looped forever, the node was restarted two or three times per run, and -c was passed again on every start. - A jar argument such as my.jar was ignored, and -j with a path lost the path before starting. - --net ignored an existing config file; --net and --release kept going after a failed download. - Linux heap sizing needed bc, and the macOS memory check failed with an expr error. - On macOS without JAVA_HOME the script derived JAVA_HOME=/usr from the /usr/bin/javac stub, and /usr/bin/java then ran itself forever; the JDK now comes from /usr/libexec/java_home. - Manifest rebuilding used a wrong URL and never ran the plugin. It now runs and, by default, rewrites manifests of 128 MB or more. - The latest release was picked in name order, clone failures were ignored, $darwin was never set, and failures exited with status 0. Platforms: - start.sh reads the Java version and architecture of its JVM. JDK 8 keeps the CMS options and JDK 17 uses the ZGC options from start.sh.simple, which is removed. ARM64 downloads the aarch64 jars and skips manifest rebuilding. - -s stops the node and the open file limit is raised to 65535, as in start.sh.simple. The default JVM_MX, used on macOS, is 12g. README and shell.md describe the single script.
Invalid node.dns.changeThreshold and node.dns.maxMergeSize now fail startup with TronError(PARAMETER_INIT); add tests for both.
Adds comments to the Java detection, the release download and signature check, the memory sizing, the manifest rebuild and the option handling of start.sh, describing what each branch does. JVM_MX drops from 12g to 9g; Linux derives the heap from the total memory, so the default applies to macOS only.
start.sh had several problems that predate this branch: - --stop matched every process whose command line contained the jar name and dropped any line containing "start", so it could kill other nodes and miss its own. The node id is now kept in <jar name>.pid; a directory with a start.log but no pid file was used by the old script and its node is still found by name. --stop also finds a node set up by --release or -cb from the parent directory, and says so when no node was started from the current one. - sh start.sh failed under dash; the script now reruns itself under bash. - an option without its value looped forever; needValue exits instead. - --disable-rewrite-manifest was only accepted as --disable-rewrite-manifes. - the first unknown option swallowed all following arguments; each unknown token is now passed on and parsing continues, with -- as an explicit end. - paths with spaces broke unquoted expansions; FullNode options are an array now. - --release downloaded straight onto FullNode.jar; it now uses a temporary file like --upgrade. - gc.log was archived before the node was stopped, and by --download. - a JDK that exists but cannot run went unnoticed, and a JVM that died right after the start was reported as running. - the manifest rebuild looked for the database next to the caller instead of the node directory, --net stored a relative config path, the gc log rotation deleted unrelated files, ulimit lowered a higher limit, and the rebuild message named logs/archive.log instead of logs/toolkit.log. shell.md documents the pid file, -- and the corrected option spelling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
maxHttpConnectNumberis reached:HttpServicesetsConnectionLimit.setIdleTimeout(10_000).start.shdownloads: TLS certificates are checked, and release jars must carry a valid GPG signature from the release key. The mainnet config is fetched from java-tron and the Nile config from nile-testnet.start.shon both x86_64 (JDK 8) and ARM64 (JDK 17): it picks the JVM options for the Java version it finds and the release jars for the JVM's architecture.start.sh.simpleis removed.start.shbugs:--stoplooping forever, the node being restarted several times per run,-cbeing passed repeatedly, a hang on macOS whenJAVA_HOMEis not set (the script derivedJAVA_HOME=/usr, so the/usr/bin/javastub ran itself forever; it now uses/usr/libexec/java_home),--stopkilling any process whose command line contains the jar name while missing a node whose arguments contain "start",sh start.shfailing under dash, an option without its value looping forever,--disable-rewrite-manifestonly being accepted misspelled, the first unknown option swallowing all following arguments, paths with spaces,--releasedownloading straight ontoFullNode.jar,gc.logbeing archived before the node was stopped, a JDK that exists but cannot run going unnoticed, and a JVM that dies right after the start being reported as running.start.shand updateshell.md.node.dns.changeThresholdornode.dns.maxMergeSizeinstead of only logging an error.KhaosDatabase.getBranch(BlockId, BlockId), which has no callers, and an unused import inManager.Why are these changes required?
start.shdownloaded withwget --no-check-certificateand checked the jar against asha256sum.txtthat releases do not publish, so a downloaded jar was never verified.start.shonly worked on x86_64 / JDK 8, since the JDK 17 JVM rejects its CMS options; ARM64 users needed a separate template.start.sh --stopfound the node withps -ef | grep -v start | grep FullNode.jar, so it could stop another node with the same jar name, or none at all. The node id is now kept in<jar name>.pid; a directory with astart.logbut no pid file was used by the previous script and its node is still found by name.This PR has been tested by:
ConnectionLimitTestpasses (20.3 s) and fails without theHttpServicechange; newArgsTestcases for an invalidnode.dns.changeThreshold/node.dns.maxMergeSizefail without theArgschange;KhaosDatabaseTest,ManagerMockTest, HTTP / JSON-RPC service tests and checkstyle pass.start.shtest suite on macOS (bash 3.2), Ubuntu 22.04 (arm64 and amd64), Debian 11 and Rocky Linux 8 covers downloads with wget and curl, verification of the real release signatures,--release/--upgrade/--download/--net, failure paths, and the JVM command for every common option; the JDK 8 command is unchanged. With real JDKs,start.shstartsFullNode.jar --helpon macOS / JDK 8 and Linux amd64 / JDK 8, andFullNode-aarch64.jar --helpon Linux arm64 / JDK 17, both withJAVACMDpreset and with the script locating the JDK itself. The final version passes 196 checks on macOS, 112 with the real network (a downloadedFullNode-aarch64.jarmatches the GitHub digest; tampered, unsigned and unverifiable downloads are removed) and 39 in an Ubuntu 26.04 / JDK 17 container, includingsh start.shunder dash, two nodes with the same jar name in different directories, a node started by the previous script, and--stopfrom the parent of a--releasesetup.Follow up
None.
Extra details
start.shnow needsgpg; the release key is fetched by fingerprint from keys.openpgp.org or keyserver.ubuntu.com.start.sh --stopis run in the directory the node was started from, or in the parent of a--release/-cbsetup; elsewhere it reports that no node was started there and exits 1. A start takes 3 seconds longer, the time used to confirm that the JVM is still running.--ends the script options; everything after it goes to FullNode unchanged.JVM_MXinstart.shis 9g. It applies on macOS; on Linux the heap is still sized from the machine's memory or-mem.start.shnow works as documented: when a LevelDB database exists andArchiveManifest.jaris missing, the verified plugin is downloaded and run before start. By default it only rewrites manifests of 128 MB or more (-m); it is skipped on ARM64 and can be turned off with-dr.