Skip to content

memory: two server processes sharing MEMORY_FILE_PATH silently discard each other's writes (the #4555 mutex is per-process) #4797

Description

@daichiyasunami-vottia

Describe the bug

#4555 fixed the read-modify-write race inside one process by serialising all six mutation methods behind KnowledgeGraphManager.withLock. That works: 20 concurrent create_entities calls against one server now keep all 20.

Two server processes sharing one MEMORY_FILE_PATH still discard each other's writes, silently. The queue is per-instance (private mutationQueue), and saveGraph writes a temp file and renames it over the target — which guarantees the file is never corrupt, and equally guarantees that the last rename replaces the other process's work wholesale.

This is the normal deployment shape for this server: MCP clients spawn their own server process, so two clients (Claude Desktop plus an editor, two checkouts/worktrees of a repo, two agents) both pointed at the default memory.json are two processes on one file.

Measurements

Harness: https://github.com/daichiyasunami-vottia/parallel-memory-mcp/blob/main/bench/bench_server_memory.mjs — spawns the server over stdio, sends N create_entities calls, then counts surviving entities. MEMORY_FILE_PATH is a fresh temp file per run.

macOS 26.5.1 (arm64, APFS), Node 25.9.0, N=20, 3 runs each — results were identical across runs.

scenario npm 2026.8.31 main @ d73f99e
serial, one process 20 / 20 20 / 20
20 concurrent, one process 1 / 20 20 / 20 ✅ (#4555)
20 concurrent, two processes, one file 1 / 20 10 / 20

Every call returns success in all three rows: ok_responses=20 error_responses=0. Nothing raises, nothing warns.

The two-process row is worth looking at closely:

survivors: entity-1,entity-3,entity-5,entity-7,entity-9,entity-11,entity-13,entity-15,entity-17,entity-19
parity   : even(procA)=0 odd(procB)=10

Process A's ten entities are not partially lost — they are entirely gone. Because each process now serialises internally, each one builds a self-consistent snapshot of the graph, and the loser's whole contribution is replaced by the winner's snapshot. Before #4555 the loss was fine-grained (one survivor out of twenty); after it, it is coarse-grained (one process's entire session). #4555 is still a clear improvement — but the failure it leaves behind is easier to mistake for "the other agent never wrote anything".

How much overlap it takes (two processes, alternating writes, main):

gap between dispatches kept
0 ms 10 / 20
10 ms 19 / 20
50 ms and above 20 / 20

So this needs the two processes' load→mutate→save windows to overlap — roughly a few milliseconds on this filesystem. That is exactly what happens when two agents save at the same moment, which for background/parallel agent use is routine rather than exotic.

To Reproduce

git clone https://github.com/daichiyasunami-vottia/parallel-memory-mcp
cd parallel-memory-mcp/bench
npm init -y && npm i @modelcontextprotocol/server-memory

# published build
node bench_server_memory.mjs multi 20

# this repo's build
cd /path/to/servers/src/memory && npm i && npm run build
SERVER_ENTRY=/path/to/servers/src/memory/dist/index.js node bench_server_memory.mjs multi 20

Expected behavior

Two processes sharing a memory file do not silently drop each other's writes. Either the write path takes a cross-process lock, or saveGraph detects that the file changed under it and re-runs the mutation against the current contents.

Sketch of the options, roughly in order of size:

  1. Advisory lock file around load→mutate→saveopen(path + '.lock', 'wx') with a stale-lock timeout, or proper-lockfile. Smallest change that closes the window; needs a stale-lock story for crashed processes.
  2. Compare-and-swap on rename — record the file's mtimeNs/size at load, re-check before rename, and retry the mutation if it moved. No lock file, no stale-lock cleanup; costs a retry loop.
  3. Append-only journal — writes become O_APPEND of one JSONL line, compaction happens separately. Removes whole-file rewriting from the common path, but is a bigger change to the file format's semantics.

I am happy to send a PR for whichever direction you prefer — (1) or (2) are both small. I did not open one straight away because the choice affects the file format's contract, which seems like a maintainer call.

Logs

Not applicable — no errors are produced in any of the runs above.

Additional context

@modelcontextprotocol/server-memory@2026.8.31 (current npm latest, published 2026-08-31) does not yet contain #4555, so the single-process race is still present for anyone installing from npm today. The table above measures both so the two are not confused.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions