Port data-repo reconciliation into the API process (closes #66)#68
Merged
Conversation
Adds a structured reconciliation state machine in `apps/api/src/store/reconcile.ts` plus a Fastify plugin that wires it between `storePlugin` and `servicesPlugin`, so the in-memory state is built from the post-reconciliation tree. This replaces the shell logic in `deploy/docker/entrypoint.sh` (see follow-up commit) and gives the future hot-reload webhook (#65) a single call to make. The state machine preserves all five outcomes from the shell version (in-sync / fast-forwarded / pushed-ahead / rebased / conflict-escaped) plus an explicit 'fetch-failed' for network blips. The conflict-escape branch name (`conflicts/<UTC>`) matches the shell format so existing operator tooling keeps working. In passing, two latent bugs from the shell version are obsoleted: 1. Single-branch clone refspec narrowness — we now always fetch with an explicit `+refs/heads/<branch>:refs/remotes/<remote>/<branch>`, so `git clone --branch X` followed by reconciling against the same X keeps working regardless of remote.origin.fetch. 2. `git rebase | sed` swallowing the rebase exit code — Node's execFile rejects the Promise on a non-zero exit, no pipe in sight. Also adds a small `dataRepoLock` decoration on Fastify (a single-slot async lock) for serializing reconciliation with future webhook-driven git operations. At boot it's uncontended; #65's handler will hold it across the fetch + in-memory rebuild. Refs #66. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The entrypoint's reconcile state machine moved into the API process (prior commit). What's left is the bits that *must* run before the Node process exists: trust the PVC, set a pseudonymous git identity for rebase committer lines, and ensure a `.git` exists by doing a full-history clone on first boot. About 190 lines of shell → ~75 (most of it now comments). The reconciliation, conflict-escape-hatch, and fetch-failure handling are all the API's job now. Also updates docs/operations/deploy.md "Boot sequence" to reflect the split — entrypoint clones, API reconciles. Refs #66. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Move the data-repo reconciliation state machine from
deploy/docker/entrypoint.shinto the Node API process so the same code can serve both boot-time and the future hot-reload webhook (#65), and so the two latent shell-pipe bugs we just hit in production are obsoleted by structured Node error handling.apps/api/src/store/reconcile.ts— the state machine, with the same five outcomes the shell had (in-sync/fast-forwarded/pushed-ahead/rebased/conflict-escaped) plus an explicitfetch-failedfor network blips. Same conflict-escape semantics — abort rebase, create + push aconflicts/<UTC>branch from pre-rebase HEAD, hard-reset local to origin.apps/api/src/plugins/reconcile.ts— Fastify plugin registered betweenstorePluginandservicesPlugin(so the in-memory state is built from the post-reconciliation tree). Decoratesfastify.dataRepoLock(a single-slot async lock) andfastify.reconcileDataRepo({branch})so Webhook endpoint for hot-reload on push to the published data branch #65's webhook can call the same code path with one line.apps/api/tests/data-repo-reconcile.test.ts— seven cases covering all five outcomes against ephemeral bare-repo "remotes", plus afetch-failedcase (bogus remote URL) and a regression test for the single-branch-clone refspec bug.deploy/docker/entrypoint.sh— trimmed from ~190 lines to ~75. What's left: trust the PVC, set a pseudonymous git identity, full-history clone on first boot. The reconciliation runs inside the API process.docs/operations/deploy.md— updated boot-sequence section + a new reconciliation-state-machine reference.Bugs obsoleted in passing
+refs/heads/<branch>:refs/remotes/<remote>/<branch>. No more silently emptyrefs/remotes/origin/Yaftergit clone --branch X.git rebase | sedswallowed the rebase exit code. Node'sexecFilerejects the Promise on non-zero exit — the pipe class of bug doesn't exist in this code.State machine outcomes tested
in-syncfast-forwardedpushed-aheadrebasedconflict-escapedfetch-failedWhat's kept in the entrypoint and why
git config --global safe.directory $CFP_DATA_REPO_PATH— must happen before any git operation in the container because the PVC may carry files owned by a different uid.git config user.name/email.openPublicStore()errors out if.gitdoesn't exist. Subsequent boots: the PVC has a clone and we justexec node. The reconciler picks it up from there.git remote set-url origin "$CFP_DATA_REMOTE"for already-cloned PVCs — keeps an operator-rotated remote URL live without forcing a re-clone.Surprises / decisions
Mutexis not exposed onRepository. We don't reach through internals; instead the plugin owns a single-slot lock at the Fastify layer (apps/api/src/lib/data-repo-lock.ts). Boot is uncontended; Webhook endpoint for hot-reload on push to the published data branch #65's webhook will hold this lock across fetch + in-memory rebuild.conflicts/YYYY-MM-DDTHH-MM-SSZshape the shell version produced, so any operator alerting onconflicts/*ref creation keeps working unchanged.receive.denyCurrentBranch=ignoreon the bare in tests — needed because the bare carriesmainas its HEAD ref and the defaultreceive.denyCurrentBranch=refusewould reject pushes. This is only on the test-rig bare; production pushes go to GitHub which doesn't have working trees.outcome: 'pushed-ahead'still returns success even when the push itself fails — same semantic the shell had. The push daemon retries on its schedule, and we don't want a transient push failure to crash the pod.Test plan
npm run -w packages/shared buildnpm run -w apps/api type-checknpm run -w apps/api test(233/233 pass, including 7 new indata-repo-reconcile.test.ts)npm run type-checkacross all workspacesnpm run lintdata-repo reconciledinfo line at bootconflict escape hatcherror line + aconflicts/<UTC>branch on originCloses #66.
🤖 Generated with Claude Code