feat(server-ng): confirm committed sends with partition and offset - #3781
Open
hubcio wants to merge 4 commits into
Open
feat(server-ng): confirm committed sends with partition and offset#3781hubcio wants to merge 4 commits into
hubcio wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3781 +/- ##
=============================================
- Coverage 75.53% 61.01% -14.53%
Complexity 969 969
=============================================
Files 1317 1318 +1
Lines 154895 155473 +578
Branches 128519 129089 +570
=============================================
- Hits 117004 94862 -22142
- Misses 34344 57147 +22803
+ Partials 3547 3464 -83
🚀 New features to boost your workflow:
|
A committed SendMessages answered with an empty body, so producers could not learn which partition a balanced or keyed batch landed in nor the offsets it was assigned. server-ng now builds a confirmation payload at commit time on the partition group primary: a count-prefixed list of entries carrying stream, topic and partition ids plus the base offset of the batch, derived from the journaled batch header so any primary draining the commit reports identical offsets. The list shape lets a future multi-partition produce grow the count without a wire change; an undecodable journal entry degrades to an empty list rather than an empty body. The SDK always returns the confirmation struct. The legacy server reports none (empty reply body); until core/server is retired that case degrades to a zeroed confirmation instead of an error, kept in one fallback marked for removal. IggyProducer returns one confirmation per chunk it split a send into; the background producer keeps returning nothing since its shard merges batches and no 1:1 mapping exists. The HTTP endpoint mirrors the tuple as JSON on the 201 reply. The PHP, Python and C++ bindings expose the confirmations as well. All three are FFI over the Rust SDK, so the changed signature reached them directly and PHP no longer compiled at all. Each wraps the reply in a language-native type mirroring the Rust shape, a response holding one confirmation per partition, so a later multi-partition produce needs no binding change either. The temporary zeroed-confirmation shape is marked in each binding alongside the fallback it stands in for. The Node SDK reads the wire itself instead of going through the Rust SDK, so it carries its own decoder. A partition reply has no committed result section, which leaves the reply body as the bare confirmation list under both classic and VSR framing. `message.send` returns that list in place of a boolean and degrades an empty body to the same zeroed confirmation; the status-only response helper that existed only to tolerate that body is gone with it.
hubcio
force-pushed
the
send_messages_confirmation
branch
from
July 31, 2026 06:30
d9f1e90 to
1bd98b6
Compare
Three paths reported a send as committed when it was not. The SDK fabricated an all-zero confirmation whenever there was nothing true to report, dispatch answered an unresolvable namespace with an empty success ack, and an unparseable reply header rendered as an empty confirmation list. None of the three is distinguishable from a genuine commit: server-ng stream and topic ids are slab keys, so zero is a real id, and the first batch offset is zero. Absence is now reported as absence. An empty body decodes to an empty list, an unresolvable namespace is denied with ResourceNotFound so HTTP keeps its 404, and the graded reply header is passed through instead of reparsed. A committed write whose confirmation cannot be read is no longer retried. The retry loop had no error-kind filter, so a decode failure raised after the server committed turned one durable write into as many copies as the retry budget allowed, on a plane that keeps no reply cache. The confirmation entry is frozen at 20 bytes. Widening it would let an older reader take its second entry from the middle of the first and return the garbage as a successful decode, so a new field needs a new response type or a version gate. The entry count may still grow freely. The producer returns one confirmation list rather than a vector of them, and ProducerSendFailed carries the prefix that did commit. The docs no longer claim replicated durability: the default config runs a single node, where that word promises a guarantee it does not provide.
A send with nothing to report was answered with an all-zero confirmation so the return shape stayed uniform. Server-ng ids are 0-based slab keys and a first batch commits at offset 0, so that entry is indistinguishable from a genuine one. The core stopped minting it, but the foreign clients still synthesized it, or documented and asserted it as the expected result. Node decoded an empty body into a zeroed entry and now yields an empty list. Python, PHP and C++ never synthesized anything. Their exposure was documentary, so the contract and the tests moved instead. Every base_offset surface now states that delivery is at-least-once, that a batch is confirmed once committed in memory rather than fsynced, and that the legacy server reports no confirmations at all. The binding tests assert an empty list rather than guarding a read. At partition 0 on a first batch a fabricated entry satisfies every plausibility check a guard could make, so the guard would pass on exactly the value being removed. Node's send-message test runs in both the legacy and the server-ng lane, so it branches on the protocol and proves a real confirmation by its offset advancing. Python and PHP now expose SendMessagesConfirmation, matching the name Node and C++ already used.
The test still required an empty legacy reply body to decode into a zeroed confirmation, which is the behaviour the SDK stopped producing, so it failed on disk. Wave 1 scoped its test run to --lib to avoid it. It now asserts the list is empty. A guarded read was rejected: at partition 0 on a first batch a zeroed entry satisfies every plausibility check such a guard could make, so it would pass on exactly the value being removed. This is the only send_confirmation test that runs in CI, which compiles the vsr-gated cases but executes with vsr off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A committed SendMessages answered with an empty body, so producers
could not learn which partition a balanced or keyed batch landed in
nor the offsets it was assigned.
server-ng now builds a confirmation payload at commit time on the
partition group primary: a count-prefixed list of entries carrying
stream, topic and partition ids plus the base offset of the batch,
derived from the journaled batch header so any primary draining the
commit reports identical offsets. The list shape lets a future
multi-partition produce grow the count without a wire change; an
undecodable journal entry degrades to an empty list rather than an
empty body.
The SDK always returns the confirmation struct. The legacy server
reports none (empty reply body); until core/server is retired that
case degrades to a zeroed confirmation instead of an error, kept in
one fallback marked for removal. IggyProducer returns one
confirmation per chunk it split a send into; the background
producer keeps returning nothing since its shard merges batches and
no 1:1 mapping exists. The HTTP endpoint mirrors the tuple as JSON
on the 201 reply.
The PHP, Python and C++ bindings expose the confirmations as well.
All three are FFI over the Rust SDK, so the changed signature
reached them directly and PHP no longer compiled at all. Each wraps
the reply in a language-native type mirroring the Rust shape, a
response holding one confirmation per partition, so a later
multi-partition produce needs no binding change either. The
temporary zeroed-confirmation shape is marked in each binding
alongside the fallback it stands in for.