Repository navigation
chdb: one long-lived server process instead of a Python process per query - #3444
Open
alexey-milovidov wants to merge 1 commit into
Open
alexey-milovidov wants to merge 1 commit into
alexey-milovidov wants to merge 1 commit into
Conversation
…uery `./start` now runs `chdb/server.py`: a single Python process with one chDB instance over `.clickbench`, serving queries on a Unix socket. Every connection is served by its own thread with its own `Session`, reused for all queries on that connection. `./query`, `./check`, `./load` and the new `./concurrent-worker` are stdlib-only clients (`chdb/client.py`). Before, every `./query` opened a `Session` on `.clickbench` in a fresh process, so concurrent queries failed with "Cannot lock file .clickbench/status" (3/300 successful queries with 4 workers in the randomized benchmark group a73b3f6e-b715-480c-b46a-2cb2fd0dbc09). The entry is now restartable (a cold run restarts the server), and the concurrent QPS test is enabled, with one connection (one session) per worker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alexey-milovidov
had a problem deploying
to
benchmark-approval
November 19, 2026 08:56 — with
GitHub Actions
Error
alexey-milovidov
deployed
to
benchmark-approval
November 19, 2026 19:18 — with
GitHub Actions
Active
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
./startnow runschdb/server.py: one long-lived Python process with a single chDB instance over.clickbench, accepting concurrent connections on a Unix socket (chdb.sock). Every connection is served by its own thread with its ownSession, reused for all queries sent over that connection:./query,./check,./loadand the new./concurrent-workerare stdlib-only clients (chdb/client.py), so they don't load chDB. The time is measured by the server aroundsession.query.Motivation: before, every
./querystarted a new Python process and opened aSessionon.clickbench, so concurrent queries failed withCannot lock file .clickbench/status. In the randomized benchmark groupa73b3f6e-b715-480c-b46a-2cb2fd0dbc09(4 workers), chdb had 3/300 successful queries.Other changes in the entry:
BENCH_RESTARTABLEis now the defaultyes: a cold run restarts the server.BENCH_CONCURRENT_WORKER=./concurrent-workerkeeps one connection, so one session, per worker, and reconnects if the watchdog restarts the server../loadgoes through the server (DROP TABLE IF EXISTS clickbench.hits SYNC, thencreate.sqlandinsert.sql), because the server holds the data directory.Tested locally with chdb 4.5.1 / chdb-core 26.11.1 on a 2M-row
generateRandomtable: 8 parallel./querycalls all succeeded; 4 concurrent workers ran ~950-990 queries each in 20 s with 0 errors; data survives a stop and start; a warm startup takes 0.3-0.45 s. A full benchmark run has not been done yet.🤖 Generated with Claude Code