fix(genesis-writer): emit six state fields the indexer reads - #491
Merged
Conversation
The indexer's create handlers read six metadata keys the writer never emitted, so each column silently landed on its default. Row-count parity cannot see this: the counts match while the columns are empty. Confirmed on a production clone (audius_discovery_2026_08_07): saves.is_save_of_repost 28,155 rows reposts.is_repost_of_repost 21,366 rows playlists.is_image_autogenerated 4,865 rows playlists.is_scheduled_release 566 rows comments.is_members_only 39 rows comments.video_url 15 rows Tracks were fixed separately in #486; these are the remaining six. All six are read with a default that matches an unset column -- MetadataBoolOr(key, false) for the booleans, MetadataString -> NULL for video_url -- so `omitempty` cannot flip a value on read, and the keys ride only on the rows that actually set them rather than on all 28M saves. That is the opposite of is_delete/is_available, which default to true on read and carry an "always serialized" comment for that reason. is_members_only is emitted verbatim: every source row that sets it is entity_type='FanClub', which is exactly what validateCommentWrite requires and what insertCommentWithState honors. The new test replays what each writer step emits through the real migration dispatcher and asserts the indexed column, rather than asserting the emitted JSON -- a key the indexer spells differently would pass the latter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rickyrombo
force-pushed
the
fix/genesis-writer-six-fields
branch
from
August 10, 2026 17:55
412fea5 to
38063c2
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug class
The indexer's create handlers read a metadata key the writer never emits, so the column silently lands on its default. Row-count parity is blind to it — the counts match while the columns are empty. Tracks were fixed separately in #486; these are the six remaining instances.
is_save_of_repostsocial_save.go:89saves.is_save_of_repostis_repost_of_repostsocial_repost.go:83reposts.is_repost_of_repostis_image_autogeneratedplaylist_create.go:94playlists.is_image_autogeneratedis_scheduled_releaseplaylist_create.go:96playlists.is_scheduled_releaseis_members_onlycomment_create.go:44comments.is_members_onlyvideo_urlcomment_create.go:45comments.video_urlEvery count re-measured on
audius_discovery_2026_08_07; the two playlist reads had shifted from the line numbers originally reported.omitempty, decided per field
The convention is that a boolean whose read-side default is true must be serialized without
omitempty(a droppedfalseflips it on read) — that is whyis_deleteandis_availablecarry an "always serialized" comment. Each of the six was checked individually:is_save_of_repostMetadataBoolOr(..., false)omitempty— safe, and keeps the key off ~28M savesis_repost_of_repostMetadataBoolOr(..., false)omitempty— sameis_image_autogeneratedMetadataBoolOr(..., false)omitempty— matches its neighboursis_album/is_private/is_stream_gatedis_scheduled_releaseMetadataBoolOr(..., false)omitempty— sameis_members_onlyMetadataBoolOr(..., false)omitempty— 39 rows carry it, none of the restvideo_urlMetadataString→nullString→NULLomitempty— absent,""andNULLare the same state here, so nothing is lostNone of the six defaults to true, so no field needed the always-serialized treatment.
video_urlis nullabletextin the source and is carried as*string, so an absent URL staysNULLrather than becoming''.is_members_onlyis emitted verbatim from the column: all 39 source rows that set it areentity_type='FanClub', which is exactly whatvalidateCommentWriterequires and whatinsertCommentWithStatehonors. (The 15video_urlrows are 11 FanClub + 4 Event.)Scan alignment
Each of the four queries touched had its SELECT list diffed pairwise against its
rows.Scanargument list, mechanically rather than by eye. Counts and every position match:s.is_save_of_repost→&s.isSaveOfRepostr.is_repost_of_repost→&r.isRepostOfRepostp.is_image_autogenerated→&p.IsImageAutogenerated(pos 8),p.is_scheduled_release→&p.IsScheduledRelease(pos 17)c.is_members_only→&c.IsMembersOnly(10),c.video_url→&c.VideoURL(11)The two playlist columns were inserted next to their semantic neighbours rather than appended, and the Scan was moved in step, so the mirrored SELECT/Scan grouping the file already had is preserved.
Testing
TestWriterEmitsStateFieldsTheIndexerReadsincmd/genesis-writer/entities_state_fields_test.gofollows theentities_comment_pin_test.gopattern: build a DP-shaped source schema, run the real writer step, then replay the emitted transactions through the real migration dispatcher (production handlers +RegisterMigrationOverrides, asindexer.gobuilds it) and assert the indexed column. Asserting the emitted JSON alone would not catch a key the indexer spells differently. Playlists and comments each include a negative-control row so a blankettruefails too.No restructuring was needed: all four metadata builders are inline closures, but each step is callable on its own (
writeSaves,writeReposts,writePlaylists,writeComments), so the whole path is exercised end to end without extracting a builder the way #486 had to.Verified red by reverting the three
entities_*.gofiles and keeping the test:All six fail on their own assertion. With the emission restored the package is green (
TestWriteCommentPins_SetsPinnedCommentID,TestWriteMutedUsers_ReachesMutedUsersTable, and the new test).go build ./cmd/genesis-writer/,gofmt -landgo vetare clean.Coordination
Rebased onto
origin/mainafter #489 landed; theentities_social.gooverlap auto-merged. No newtime.RFC3339formatting is introduced, so nothing here interacts with the.UTC()guard in #488.🤖 Generated with Claude Code