Repository navigation
memory: reduce repeated topic curation with frozen preparation - #3282
Merged
Hillary Mutisya (hillary-mutisya) merged 2 commits intoNov 11, 2026
Merged
Conversation
Add opt-in prior deltas, compact wire IDs and two-worker frozen preparation with serialized, validated reconciliation and ordered watermark receipts. Harden retained-input replay with model bindings, absolute deadlines, validated partial progress and the corrected diagnostic organization handoff. Preserve default behavior and native receipt validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep deadline cancellation artifacts and persistence failures unchanged while bringing the replay lifecycle within the committed-branch complexity budget. Validation: full workspace build and all 43 replay/options tests pass; the 169 service tests passed on the unchanged service implementation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Motivation
Serial topic curation repeatedly sends and rewrites already-admitted concepts. Speculative preparation can also become invalid when earlier decisions change its prior-concept frontier. Prepare independent raw packets against frozen inputs instead, then reconcile their proposals against current admitted concepts before installing a complete window.
Behavior and configuration
All three new treatments are off by default:
TYPEAGENT_MEMORY_TOPIC_PRIOR_DELTASTYPEAGENT_MEMORY_TOPIC_COMPACT_IDSTYPEAGENT_MEMORY_TOPIC_FROZEN_PREPARATIONThese are also programmatic
topicCurationsettings and replay CLI switches. Effective treatment settings participate in policy/replay identity. Watermark continuation accepts explicit frozen two-worker execution, not ordinary speculative concurrency or the native Decision engine. Source correction/removal, forgetting, eligibility, and current-publication checks remain fail-closed.Automatic maintenance may recapture a changed topic head once during admission; repeated churn still fails stale. Explicit callers retain strict expected-head semantics.
The replay organization handoff drops captured native packet-receipt assertions from its cloned diagnostic input. It does not disable production receipt validation or certify native watermark installation.
Reviewer guide
Validation
Windows, Node 26.13.0, standalone pnpm 11.9.0. Final reviewed head:
1748afb8767aa3ed86780da91f34f621dee368ec. The branch starts from refreshedmain; unrelated model-migration work is already merged and is not included here.pnpm run buildfromtsThe service suites are
topicWorkReuse,topicBuilds,topicSnapshot,topicCuration,topicGeneration,topicLookup,topicFrozenPreparation,topicCurationScheduler, andviewHostOptions, run withpnpm run jest-esm --runInBand --runTestsByPathagainst their correspondingdist\test\*.spec.jspaths.Replay tests and PR gates, from
ts:Tests cover exact receipt replay and cold resume, append/correction/forget behavior, audited-prior preservation, unknown compact IDs and bounded repair, sibling drainage, deadline enforcement before initialization/repairs/organization, metadata redaction/binding drift, and diagnostic organization without weakening native receipt checks. The captured-watermark regression is awaited so its lifetime cannot escape its parent test.
Live diagnostic results
One sequential, fixed-order control/candidate pair reused 2,853 raw topic nodes from 45 previously ingested chats; no new ingestion. Both arms used the same configured GPT-6 Luna model, unchanged provider-default reasoning, and independently matched 300-second HTTP request budgets under one unchanged absolute deadline. Earlier 60-second timeout failures were preserved, not mixed into this pair.
Candidate completed curation and actual organization in 40m03s. Curation latency was 48.8% lower, with 25.4% fewer calls and 30.8% fewer completion tokens; prompt tokens increased 0.7%.
Control's initial organization and bounded repair both returned hierarchy cycles, so the original control arm failed. A separate 1m03s, two-call live organization retry succeeded from its exact completed checkpoint, with zero repeated curation calls and no overlap with candidate inference. The original failed attempt remains a failure, not an uninterrupted successful pipeline.
Both final snapshots passed native snapshot/curation validation and accounted for all 2,853 inputs, with zero missing/duplicate/unknown assignments. Measurements were collected on the development snapshot before publication onto refreshed main; the final PR tree is rebuilt and offline-tested separately. No additional inference was authorized or started for PR preparation.
Risks, limitations and rollout