Skip to content

fix(genesis-writer): emit email_owner_user_id so encrypted emails index - #490

Merged
rickyrombo merged 1 commit into
mainfrom
fix/genesis-writer-email-owner
Aug 10, 2026
Merged

fix(genesis-writer): emit email_owner_user_id so encrypted emails index#490
rickyrombo merged 1 commit into
mainfrom
fix/genesis-writer-email-owner

Conversation

@rickyrombo

Copy link
Copy Markdown
Contributor

All 4,218 encrypted emails and their 9,282 access grants are lost in a migration today.

encryptedEmailHandler takes the owner from the metadata key email_owner_user_id, not from the transaction's user_id:

ownerID, ok := params.MetadataInt64("email_owner_user_id")
if !ok { return NewValidationError("missing required field: email_owner_user_id") }

encryptedEmailMetadata had exactly three fields — encrypted_email, access_grants, created_at — so every EncryptedEmail/AddEmail transaction was rejected. These do reach em_rejected, but as thousands of anonymous rejections among millions of transactions, which is why it went unnoticed.

Every required key, not just the one

Reading the handler end to end turns up a second key of the same class. access_grants was tagged omitempty, and parseAccessGrants rejects the transaction when the key is absent:

grants, ok := params.MetadataJSON("access_grants")
if !ok { return nil, NewValidationError("missing required field: access_grants") }

The handler cannot tell "absent" from "no grants", so omitempty is dropped and a nil slice is serialized as [] rather than omitted (or as null, which fails the subsequent []any assertion with "access_grants must be a list"). Every email in the snapshot has at least one grant, so this one is latent — but it is the identical failure mode, one row away from firing.

The per-grant keys (receiving_user_id, grantor_user_id, encrypted_key) were already emitted correctly, and the snapshot has no NULL or empty values in any of them.

Signer

Neither email handler calls ValidateSigner, and there is no email validation at the chain layer or migration override, so the signer is not what was blocking these. The step still joins users and signs as the owner, matching every other entity writer and the convention stated in migration.go — the rows should be attributable, and should stay valid if that check is ever added.

The join also means an email whose owner is not a current user with a wallet is skipped rather than emitted unattributable. Verified against the production snapshot: all 4,218 owners are current users with a non-empty wallet, so nothing is dropped.

EmailAccess

The audit's flag is a genuine mismatch, and a worse one. The orphan-grant step emitted EmailAccess/Create with the grant fields at the top level, while emailAccessHandler is registered for EmailAccess/Update and reads the same {email_owner_user_id, access_grants} shape as the create path. Wrong action and wrong shape: those transactions matched no handler at all, so unlike the emails above they would not even have surfaced in em_rejected.

Fixed here rather than deferred — same file, same feature. It has no effect on the snapshot: every email_access row there has an encrypted_emails parent, so the step's NOT EXISTS filter selects zero rows.

One caveat worth recording: emailAccessHandler also requires the grantor to already hold access to the email, which an orphan's first grant cannot satisfy — there is no parent row to have granted from. Correcting the shape does not change that, and weakening the handler would be the wrong fix. What it does buy is that if orphan grants ever appear they reach the handler and are accepted or rejected on their merits, instead of vanishing to a routing mismatch. The code comment says so.

Snapshot verification

Against audius_discovery_2026_08_07:

encrypted_emails 4,218
email_access 9,282
emails with NULL owner id 0
emails whose owner is a current user with a wallet 4,218
emails with zero grants 0
orphan email_access rows (no parent email) 0
grants with NULL/empty encrypted_key or ids 0

Test

TestWriteEncryptedEmails_IndexesWithGrants drives the writer step against a source snapshot and replays what it emits through the real handler — the only level at which this was visible. The handler-side email tests in pkg/etl all pass today while no transaction ever reaches them.

Reverting just the metadata field fails it with the production symptom:

entities_email_test.go:147: metadata email_owner_user_id = 0, want 401
entities_email_test.go:174: indexing the emitted encrypted email: missing required field: email_owner_user_id
--- FAIL: TestWriteEncryptedEmails_IndexesWithGrants

Reverting the file wholesale also fails the wallet-signer and skip assertions, and TestWriteEmailAccess_EmitsHandlerShape fails with emitted 0 EmailAccess/Update transactions, want 1.

With the fix, both pass, alongside the existing writer and ETL suites:

--- PASS: TestWriteCommentPins_SetsPinnedCommentID (0.35s)
--- PASS: TestWriteEncryptedEmails_IndexesWithGrants (0.49s)
--- PASS: TestWriteEmailAccess_EmitsHandlerShape (0.04s)
ok  github.com/OpenAudio/go-openaudio/pkg/etl/processors/entity_manager  114.841s

Note on #488

#488 adds .UTC() to timestamp formatting in this file. Both Format(time.RFC3339) calls here already carry .UTC(), so the two changes agree on those lines.

🤖 Generated with Claude Code

All 4,218 encrypted emails and their 9,282 access grants are lost in a
migration today. encryptedEmailHandler takes the owner from the metadata
key email_owner_user_id rather than from the transaction's user_id, and
encryptedEmailMetadata had exactly three fields -- encrypted_email,
access_grants, created_at -- so every EncryptedEmail/AddEmail transaction
was rejected with "missing required field: email_owner_user_id". They do
reach em_rejected, but as thousands of anonymous rejections among
millions of transactions, which is why this went unnoticed.

access_grants gets the same treatment for the same reason: the handler
rejects the transaction when the key is absent and cannot tell "absent"
from "no grants", so `omitempty` is dropped and a nil slice is serialized
as []. Every email in the production snapshot has at least one grant, so
this is latent rather than active, but it is the identical failure mode.

The step now joins users for the owner's wallet and signs as the owner,
matching every other entity writer. The email handlers do not call
ValidateSigner, so this is not what unblocks them -- it is what keeps the
migration's own convention intact, so the rows are attributable and stay
valid if that check is ever added. The join also means an email whose
owner is not a current user with a wallet is skipped rather than emitted
unattributable; on a production clone all 4,218 qualify, so nothing is
dropped.

The orphan-grant step had a second, latent mismatch: it emitted
EmailAccess/Create with the grant fields at the top level, while
emailAccessHandler is registered for EmailAccess/Update and reads the
same {email_owner_user_id, access_grants} shape as the create path. Those
transactions matched no handler at all, so unlike the emails above they
would not even have surfaced in em_rejected. It is corrected here rather
than left, since it is the same file and the same feature. It has no
effect on the snapshot: every email_access row there has an
encrypted_emails parent, so the step selects nothing.

The test drives the writer step against a source snapshot and replays
what it emits through the real handler, which is the only level at which
this was visible -- the handler-side email tests all passed while no
transaction reached them. Reverting the metadata field alone fails it
with "indexing the emitted encrypted email: missing required field:
email_owner_user_id".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rickyrombo
rickyrombo merged commit b7c3597 into main Aug 10, 2026
5 checks passed
@rickyrombo
rickyrombo deleted the fix/genesis-writer-email-owner branch August 10, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant