Skip to content

fix(genesis-writer): emit six state fields the indexer reads - #491

Merged
rickyrombo merged 1 commit into
mainfrom
fix/genesis-writer-six-fields
Aug 10, 2026
Merged

fix(genesis-writer): emit six state fields the indexer reads#491
rickyrombo merged 1 commit into
mainfrom
fix/genesis-writer-six-fields

Conversation

@rickyrombo

Copy link
Copy Markdown
Contributor

The bug class

The indexer's create handlers read a metadata key the writer never emits, so the column silently lands on its default. Row-count parity is blind to it — the counts match while the columns are empty. Tracks were fixed separately in #486; these are the six remaining instances.

metadata key indexer reads at target column source rows
is_save_of_repost social_save.go:89 saves.is_save_of_repost 28,155
is_repost_of_repost social_repost.go:83 reposts.is_repost_of_repost 21,366
is_image_autogenerated playlist_create.go:94 playlists.is_image_autogenerated 4,865
is_scheduled_release playlist_create.go:96 playlists.is_scheduled_release 566
is_members_only comment_create.go:44 comments.is_members_only 39
video_url comment_create.go:45 comments.video_url 15

Every count re-measured on audius_discovery_2026_08_07; the two playlist reads had shifted from the line numbers originally reported.

omitempty, decided per field

The convention is that a boolean whose read-side default is true must be serialized without omitempty (a dropped false flips it on read) — that is why is_delete and is_available carry an "always serialized" comment. Each of the six was checked individually:

field read-side default conclusion
is_save_of_repost MetadataBoolOr(..., false) omitempty — safe, and keeps the key off ~28M saves
is_repost_of_repost MetadataBoolOr(..., false) omitempty — same
is_image_autogenerated MetadataBoolOr(..., false) omitempty — matches its neighbours is_album / is_private / is_stream_gated
is_scheduled_release MetadataBoolOr(..., false) omitempty — same
is_members_only MetadataBoolOr(..., false) omitempty — 39 rows carry it, none of the rest
video_url MetadataStringnullStringNULL omitempty — absent, "" and NULL are the same state here, so nothing is lost

None of the six defaults to true, so no field needed the always-serialized treatment. video_url is nullable text in the source and is carried as *string, so an absent URL stays NULL rather than becoming ''.

is_members_only is emitted verbatim from the column: all 39 source rows that set it are entity_type='FanClub', which is exactly what validateCommentWrite requires and what insertCommentWithState honors. (The 15 video_url rows are 11 FanClub + 4 Event.)

Scan alignment

Each of the four queries touched had its SELECT list diffed pairwise against its rows.Scan argument list, mechanically rather than by eye. Counts and every position match:

  • saves — 7 cols / 7 args, s.is_save_of_repost&s.isSaveOfRepost
  • reposts — 7 / 7, r.is_repost_of_repost&r.isRepostOfRepost
  • playlists — 28 / 28, p.is_image_autogenerated&p.IsImageAutogenerated (pos 8), p.is_scheduled_release&p.IsScheduledRelease (pos 17)
  • comments — 11 / 11, c.is_members_only&c.IsMembersOnly (10), c.video_url&c.VideoURL (11)

The two playlist columns were inserted next to their semantic neighbours rather than appended, and the Scan was moved in step, so the mirrored SELECT/Scan grouping the file already had is preserved.

Testing

TestWriterEmitsStateFieldsTheIndexerReads in cmd/genesis-writer/entities_state_fields_test.go follows the entities_comment_pin_test.go pattern: build a DP-shaped source schema, run the real writer step, then replay the emitted transactions through the real migration dispatcher (production handlers + RegisterMigrationOverrides, as indexer.go builds it) and assert the indexed column. Asserting the emitted JSON alone would not catch a key the indexer spells differently. Playlists and comments each include a negative-control row so a blanket true fails too.

No restructuring was needed: all four metadata builders are inline closures, but each step is callable on its own (writeSaves, writeReposts, writePlaylists, writeComments), so the whole path is exercised end to end without extracting a builder the way #486 had to.

Verified red by reverting the three entities_*.go files and keeping the test:

--- FAIL: TestWriterEmitsStateFieldsTheIndexerReads
    saves.is_save_of_repost = false, want true (the source row sets it;
      metadata was {"created_at":"2025-06-01T12:00:00Z","is_delete":false})
    reposts.is_repost_of_repost = false, want true (the source row sets it;
      metadata was {"created_at":"2025-06-01T12:00:00Z","is_delete":false})
    playlist 9301 is_image_autogenerated = false, want true
    playlist 9301 is_scheduled_release = false, want true
    comment 9401 is_members_only = false, want true
    comment 9401 video_url = <nil>, want "https://example.com/clip.mp4"

All six fail on their own assertion. With the emission restored the package is green (TestWriteCommentPins_SetsPinnedCommentID, TestWriteMutedUsers_ReachesMutedUsersTable, and the new test). go build ./cmd/genesis-writer/, gofmt -l and go vet are clean.

Coordination

Rebased onto origin/main after #489 landed; the entities_social.go overlap auto-merged. No new time.RFC3339 formatting is introduced, so nothing here interacts with the .UTC() guard in #488.

🤖 Generated with Claude Code

The indexer's create handlers read six metadata keys the writer never
emitted, so each column silently landed on its default. Row-count parity
cannot see this: the counts match while the columns are empty.

Confirmed on a production clone (audius_discovery_2026_08_07):

  saves.is_save_of_repost           28,155 rows
  reposts.is_repost_of_repost       21,366 rows
  playlists.is_image_autogenerated   4,865 rows
  playlists.is_scheduled_release       566 rows
  comments.is_members_only              39 rows
  comments.video_url                    15 rows

Tracks were fixed separately in #486; these are the remaining six.

All six are read with a default that matches an unset column --
MetadataBoolOr(key, false) for the booleans, MetadataString -> NULL for
video_url -- so `omitempty` cannot flip a value on read, and the keys
ride only on the rows that actually set them rather than on all 28M
saves. That is the opposite of is_delete/is_available, which default to
true on read and carry an "always serialized" comment for that reason.

is_members_only is emitted verbatim: every source row that sets it is
entity_type='FanClub', which is exactly what validateCommentWrite
requires and what insertCommentWithState honors.

The new test replays what each writer step emits through the real
migration dispatcher and asserts the indexed column, rather than
asserting the emitted JSON -- a key the indexer spells differently would
pass the latter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rickyrombo
rickyrombo force-pushed the fix/genesis-writer-six-fields branch from 412fea5 to 38063c2 Compare August 10, 2026 17:55
@rickyrombo
rickyrombo merged commit a708bc1 into main Aug 10, 2026
5 checks passed
@rickyrombo
rickyrombo deleted the fix/genesis-writer-six-fields branch August 10, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant