Skip to content

memory: add a scale-correctness round-trip test (5,000 entities, exact read-back) #5540

Open

Description

@reservoir-cmyk

Which server?

memory (@modelcontextprotocol/server-memory)

Protocol era

Modern (2026-07-28)

MCP client

No response

The problem

This proposes one test for the memory server: not a benchmark, a correctness pin at scale. The existing scale coverage is small-scale checks around correlated search; nothing verifies that what is written at thousands-of-records scale reads back exactly — no dropped records, no observations attached to the wrong entity, no relations losing endpoints — when read through a fresh manager instance (the restart equivalent).

Case

Create 5,000 entities in batches of 500 (two observations each) plus 999 chained relations, then read the graph back through a new KnowledgeGraphManager and assert: entity and relation counts, the last entity's observations verbatim, a unique-observation search hit, and the on-disk line count matching the record count. Given the JSONL one-record-per-line format and the atomic full-rewrite save, the round trip should be the identity; if some scale breaks it, that is a correctness problem before it is a performance one. (Throughput observations are deliberately not part of this proposal.)

Solution you have in mind (optional)

Proposed test

For src/memory/__tests__/knowledge-graph.test.ts (a scale describe) or a new file:

it(
  "reads back exactly what was written at 5,000-entity scale",
  async () => {
    const writer = new KnowledgeGraphManager(testFilePath);
    const TOTAL = 5000;
    const BATCH = 500;
    for (let start = 0; start < TOTAL; start += BATCH) {
      const batch: Entity[] = Array.from({ length: BATCH }, (_, i) => {
        const n = start + i;
        return {
          name: `entity-${n}`,
          entityType: "bulk",
          observations: [`obs-${n}`, `marker-${n % 7}`],
        };
      });
      await writer.createEntities(batch);
    }
    const relations: Relation[] = Array.from({ length: 999 }, (_, i) => ({
      from: `entity-${i}`,
      to: `entity-${i + 1}`,
      relationType: "next",
    }));
    await writer.createRelations(relations);

    const reader = new KnowledgeGraphManager(testFilePath);
    const graph = await reader.readGraph();
    expect(graph.entities).toHaveLength(TOTAL);
    expect(graph.relations).toHaveLength(999);
    const last = graph.entities.find((e) => e.name === "entity-4999");
    expect(last?.observations).toEqual(["obs-4999", `marker-${4999 % 7}`]);

    const found = await reader.searchNodes("obs-4242");
    expect(found.entities.map((e) => e.name)).toEqual(["entity-4242"]);

    const lines = (await fs.readFile(testFilePath, "utf-8"))
      .split("\n")
      .filter((l) => l.trim() !== "");
    expect(lines).toHaveLength(TOTAL + 999);
  },
  120_000,
);

Which MCP feature does it demonstrate? (optional)

No response

Alternatives or workarounds you have tried

Closest hit checked: #3137 concerns an open_nodes bug, not a duplicate of this test proposal. No workaround covers this — no existing test verifies exact read-back at 5,000-entity scale.

Already built it locally?

Verification

Ran against a byte-identical copy of v2/main @ 6cfff62: the case passed (the round trip itself took ~0.6 s on the test machine; the generous timeout is there to keep it non-flaky on slow CI, not as a performance claim). Full memory suite: 241 passed + 1 skipped.

Prompt used

"In src/memory, add a scale-correctness test (not a benchmark): create 5,000 entities in batches of 500 with two observations each, plus 999 chained relations, then read the graph back through a FRESH KnowledgeGraphManager instance and assert exact equality of counts, the last entity's observations, a unique-observation search hit, and the on-disk line count. It should complete in about a second; give it a generous timeout so it stays non-flaky on slow CI."

Before you submit

  • I searched existing issues and this is not a duplicate.
  • This is a request for this repository or one of its servers, not a new server, the MCP specification or an SDK.

Activity

  1. added
    server-memoryReference implementation for the Memory MCP server - src/memory
    choreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior change
    and removed
    enhancementNew feature or request
    on Nov 18, 2026
  2. cliffhall commented on Nov 18, 2026

    @cliffhall
    Member

    Triage: Priority Low (total 2)

    • Severity 1: test-only proposal pinning current behavior; no defect reported
    • Urgency 1: no time pressure; nothing waits on it
    • Bonuses: none

    Relabeled chore: a test addition with no user-facing behavior change.

    Board: Servers V2 (#43), Status Incoming, awaiting maintainer review (no milestone yet).

  3. added this to the v2.6.0 milestone on Nov 18, 2026
  4. self-assigned this
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

choreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior changeserver-memoryReference implementation for the Memory MCP server - src/memoryv2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions