Skip to content

test(memory): pin exact read-back at 5,000-entity scale - #5625

Open
cliffhall wants to merge 1 commit into
v2/mainfrom
v2/chore/5540-memory-scale-round-trip-test
Open

cliffhall wants to merge 1 commit into
v2/mainfrom
v2/chore/5540-memory-scale-round-trip-test

Conversation

@cliffhall

@cliffhall cliffhall commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Closes #5540

Description

Adds one new test file, src/memory/__tests__/scale-round-trip.test.ts: a correctness pin at scale, not a benchmark. It writes 5,000 entities in ten batches of 500 (two observations each, one holding multi-byte characters) and 999 chained relations through an MCP client, then reads them back through a second connection (a second server with its own KnowledgeGraphManager, so nothing the writer held in memory: the restart equivalent) and compares every record:

  • read_graph with no options returns exactly the 5,000 entities and 999 relations, in order.
  • search_nodes for one unique observation returns exactly that entity and the two relations that touch it.
  • The JSONL file holds one record per line: the entities, then the relations, each equal to what was written.
  • Each create_entities / create_relations call reports exactly its own batch as created.

Two departures from the test as the issue proposed it, both to make it prove what its title claims:

  • Exact comparison instead of counts and one sample. A count cannot see an observation attached to the wrong entity or a relation with a swapped endpoint.
  • Through the client harness (__tests__/helpers.ts, unchanged) instead of KnowledgeGraphManager directly. A default cap or page size would be added at the tool layer, which is also where a result this large meets the SDK's schema validation and serialization. The reasoning is in the file's purpose header.

Server Details

  • Server: memory
  • Changes to: tests only (one new file). No source, README or published-package change.

Motivation and Context

The rest of the memory suite works on graphs of a few dozen records at most (the largest loops create 20 to 25), so on small files. Here the graph file is about 600 KB, several times a 64 KiB stream chunk, written by eleven full rewrites of a growing file. A change that only breaks past a size no small test reaches passes the rest of the suite; see the probes below.

How Has This Been Tested?

No client-observable surface changes, so the evidence is a targeted probe: two scale-only defects planted in src/memory/index.ts (then reverted), each run against the existing suite and against the new file.

Planted defect Existing suite (17 files, 295 tests; stdio smoke excluded) New file
readGraph returns only the first 1,000 entities by default 295 passed read_graph returns every entity and relation… fails
saveGraph drops the record straddling each 64 KiB boundary of the file 295 passed fails in setup

Durations (setup plus all three tests, vitest's tests figure):

Where Duration
Local, Apple-silicon Mac, Node 26, vitest run, 8 runs 277 to 424 ms, plus one 958 ms run on a loaded machine
Local, same machine, vitest run --coverage, 3 runs 354 to 407 ms
The issue's original manager-level version, same machine, 5 runs 175 to 454 ms
CI at 7d0c78a, Test memory (Windows) 1,092 ms for the file (suite: 5.73 s wall)
CI at 7d0c78a, Coverage memory (Linux, instrumented) 1,065 ms
CI at 7d0c78a, Validate memory (Node 24) (Linux) 719 ms

The setup hook carries an explicit 30 s timeout, justified in a comment with the measured duration: the default 5 s is thin headroom for eleven locked rewrites on a slow Windows runner (#5570).

npm run local:gate exits 0.

Breaking Changes

None.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

None of the four: a test-only chore.

Checklist

  • I have read the MCP Protocol Documentation (not applicable: no protocol feature is touched; the test uses existing tools through the existing harness)
  • My changes follow MCP security best practices (not applicable: test-only; it writes only to an OS temp directory it removes)
  • I have updated the server's README accordingly (not applicable: no tool, resource, prompt, configuration or environment variable changes)
  • I have added a changeset (npm run changeset) if this changes what a TypeScript server publishes (not applicable: tests are not published)
  • I have tested this with an LLM client (not applicable: no server-facing change; the targeted probe above is the evidence)
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling (not applicable: test-only; the setup closes its writer in finally and the temp directory is removed in afterAll)
  • I have documented all environment variables and configuration options (not applicable: none added)

🤖 Generated with Claude Code

Add a protocol-level round-trip test: 5,000 entities in ten batches and 999
chained relations written through one client, then read back through a second
connection and compared record for record (read_graph, search_nodes and the
JSONL file). No other test stores more than a few dozen records, so a default
cap on a read tool or a chunk-boundary defect in the file path would pass the
rest of the suite.

Closes #5540

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall cliffhall added the v2 label Nov 18, 2026
@changeset-bot

changeset-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 7d0c78a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

memory: MCP interface diff

Base: 7f383a8e0f34 (Merge pull request #5597 from modelcontextprotocol/changeset-release/v2/main)
Head: the build in this checkout, at 763aa9300e69 (Merge 7d0c78a into 7f383a8)

Diffed because it was named (--server).

✅ No interface changes detected.

Tools Prompts Resources Resource templates
Base 9 0 1 0
Head 9 0 1 0

Updated by this run for 7d0c78ad44c55d4da7a62a3ff8b77af2001f714d.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the added memory-server scale test.

No open issues 🎉

What changed in this PR

Adds a test-only correctness check for the memory server at 5,000-entity scale, addressing #5540 without changing published behavior.

Changes:

  • Writes 5,000 entities and 999 relations through an MCP client, verifying each creation response.
  • Uses a fresh server connection to check exact graph read-back and a unique-observation search result.
  • Verifies every JSONL record on disk and gives setup an explicit 30-second timeout for slower runners.
File Description
src/​memory/​__tests__/​scale-round-trip.test.ts Adds scale coverage for exact persistence, fresh-connection reads, and search results.

Review effort: Balanced

@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 1 (review of 7d0c78a): clean. "Approval recommended", no comments and no suppressed comments, so there is nothing to answer and no further round is requested. The review loop ends here on the first clean round.

CI at 7d0c78a is green. The new file's reported duration: 1,092 ms on Test memory (Windows), 1,065 ms on Coverage memory, 719 ms on Validate memory (Node 24), against a 30 s setup timeout. The PR body's table carries these.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory: add a scale-correctness round-trip test (5,000 entities, exact read-back)

2 participants