Skip to content

Reduce journal edit and sync I/O amplification - #54

Merged
RCmerci merged 4 commits into
mainfrom
fix/journal-io-amplification
Oct 8, 2026
Merged

RCmerci merged 4 commits into
mainfrom
fix/journal-io-amplification

Conversation

@RCmerci

@RCmerci RCmerci commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Small Capture/edit/status operations rewrote the entire durable outbox and read unrelated page members for confirmation. Graph and sync notification bursts also replaced in-flight work and reread hidden media roots. This change reduces those writes and queries while retaining durable completion, snapshot consistency, and conservative recovery.

  • Persist outbox row deltas using stable UUID keys and immutable ordering, with independent revision metadata, transactional migration/CAS, and receipts/checkpoints in the same commit. Only publish the new durable baseline after COMMIT. Existing legacy rows migrate under the writer lock; older readers/writers fail closed after migration.
  • Confirm Capture/edit/status with a Worker query for the target, page and paginated direct children from one snapshot. Keep existing page-tree interests and child page/tag/revision caches. Use 50-item pages, strict versions, and bounded stale restarts.
  • Coalesce metadata requests per root. Refresh active root/asset/preview owners, mark hidden roots dirty for activation, discard stale global cursors, and retain one dirty followup. Separate idempotent offline configuration from actual graph invalidation; retain committed demand until a fresh scan completes. Current-request ownership fences pull-resync routing.
  • Keep maximum sync pull demand in Core with one connection-scoped owner through durable application. Handle partial/empty progress, delayed retry, timeout and send failure without advancing the checkpoint to an advertised head.

Validation: public production-owner regressions failed before implementation, then passed; final dune build @all app/native_embed.exe.o and dune runtest --force both passed. Focused evidence includes 46 storage cases, 85 graph Runtime cases, 10 actual Worker cases, 86 sync cases, and 24 new media/offline owner cases. SQL audit counts for N=1000 append operations fall from 1,000,000 queue-row changes to 1000; Capture confirmation with 201 children uses five pages despite 1000 unrelated siblings; 64 retained media roots with two active issue two reads, with 32 further notifications adding no in-flight request; opening + Hello + 100 Changed effects fall from 102 pulls to one.

These are synthetic logical SQL/request counters, not physical WAL/SSD bytes or device timings. Outbox encoding/map work still traverses N records; children membership digests and legacy caches retain existing CPU costs. Active media roots conservatively refresh because block/structure/known-reference summaries cannot prove complete negative or ancestor dependencies. Offline clean stale failure remains visible with committed demand retained until graph invalidation/configuration replacement. The private Application late-resync binding was reviewed in source and compiled; public Runtime tests do not directly execute that binding.

No protected spec/*.mli or Dune changes. The four new implemented decision documents validate; spec-dev-tool check --all still reports missing sections in two unchanged historical decisions. F4/F6/F7, real graph/account/phone/GUI testing, and merge are outside this change.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T03:43:52.840823Z a3ef4cc PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@RCmerci
RCmerci merged commit fc83377 into main Oct 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant