Skip to content

test(memory): pin that no tool removes or repairs a schema-invalid record - #5619

Open
cliffhall wants to merge 1 commit into
v2/mainfrom
v2/chore/5541-memory-invalid-record-removal-test
Open

cliffhall wants to merge 1 commit into
v2/mainfrom
v2/chore/5541-memory-invalid-record-removal-test

Conversation

@cliffhall

Copy link
Copy Markdown
Member

Closes #5541

Description

Adds one test file, src/memory/__tests__/invalid-record-removal.test.ts, that pins what a stored record failing the schema costs once it is in the memory file. #4885 made the server skip such a line on read and write it back verbatim on every save; persistence.test.ts and knowledge-graph.test.ts already pin those two facts, and only for a create_entities write. Nothing pinned the consequence: the record is out of every tool's reach.

Eight cases, each driven through an SDK Client over the in-memory transport and asserting on the tool result and the file's exact bytes, from a file seeded with two valid entities, one valid relation, an entity record with no entityType and a relation record with no relationType:

Tool What it does with the invalid record
read_graph, search_nodes, open_nodes Do not show it; the file is untouched
delete_entities Deleted 0 of 1 entities. Not found: Broken; the line survives it and a later unrelated write
delete_observations Deleted 0 of 1 observations. Entities not found: Broken; file unchanged
add_observations Tool error Entity with name Broken not found; file unchanged
create_relations Tool error Entity with name Broken not found; file unchanged
create_entities The name reads as free, so a second record is stored beside the invalid line; deleting by that name then removes only the new one
delete_relations The invalid relation cannot be named (omitting relationType is an input validation error) and matches no call
delete_entities (cascade) Removes the valid relations of the entities it deletes; the invalid relation that names them is left behind

The issue proposed a single case calling KnowledgeGraphManager directly, added to knowledge-graph.test.ts. This differs in two ways, on purpose:

  • Protocol-level. The claim is about tools, and the "not found" wording a client reads is built in the tool handlers, which a direct manager call skips.
  • Every write tool. The title says "any tool", so each of the six write tools is exercised rather than delete_entities alone. That turned up the create_entities behavior above, which the issue did not describe.

The tests take no position on whether a remediation tool should exist. They state today's contract, and a change that adds one changes these assertions visibly.

Server Details

  • Server: memory
  • Changes to: tests only. No source, README or interface change.

Motivation and Context

#5541. The preservation behavior from #4885 is deliberate; its consequence was unpinned and undocumented. The README's silence on it is filed separately as #5609 rather than widened into this PR.

How Has This Been Tested?

No client-observable surface changes, so this carries a targeted probe rather than client evidence.

  • npm test -w src/memory -- __tests__/invalid-record-removal.test.ts: 8 passed.
  • Planted defect: with saveGraph changed to drop the unreadable lines instead of writing them back (the pre-memory: unreadable lines are deleted by the next write (regression from #4717, unreleased) #4885 behavior, and what any tool that removed them would look like on disk), 5 of the 8 cases fail: every case in which a save happens. The 3 that still pass are the ones in which nothing is written (the read tools, and the two add tools that throw before saving). The change was reverted; src/memory/index.ts is untouched in this PR.
  • npm run local:gate exited 0, including the memory coverage stage and the interface diff ("No interface changes detected", 9 tools and 1 resource on both sides).

Breaking Changes

None.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

None of the four: a test-only chore.

Checklist

  • I have read the MCP Protocol Documentation (not applicable: no protocol feature is added or changed)
  • My changes follow MCP security best practices (not applicable: tests only)
  • I have updated the server's README accordingly (not applicable: no behavior changes; the README gap this surfaced is memory: README does not say that unreadable lines are skipped, kept on every write, and fixable only by hand #5609)
  • I have added a changeset (npm run changeset) if this changes what a TypeScript server publishes (not applicable: nothing published changes)
  • I have tested this with an LLM client (not applicable: no server-facing change; see the probe above)
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling (not applicable: tests only; the expected stderr lines are suppressed)
  • I have documented all environment variables and configuration options (not applicable: none added)

Additional context

The test file opens with a purpose header giving the reasoning for the protocol-level harness. It uses __tests__/helpers.ts unchanged and an OS temp directory per test.

🤖 Generated with Claude Code

…cord

A stored record that fails the schema is skipped on read and written back
verbatim on every save (#4885). Pin the consequence through the protocol:
the read tools do not show it, the delete tools report it as not found, the
add tools refuse it as missing, create_entities under its name stores a
second record beside it, and an invalid relation record matches no
delete_relations call and outlives the entities it names.

Closes #5541

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall cliffhall added the v2 label Nov 18, 2026
@changeset-bot

changeset-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: fb2dae7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cliffhall
cliffhall requested a balanced review from Copilot November 18, 2026 18:39
@github-actions

Copy link
Copy Markdown
Contributor

memory: MCP interface diff

Base: 7f383a8e0f34 (Merge pull request #5597 from modelcontextprotocol/changeset-release/v2/main)
Head: the build in this checkout, at 1291d505c897 (Merge fb2dae7 into 7f383a8)

Diffed because it was named (--server).

✅ No interface changes detected.

Tools Prompts Resources Resource templates
Base 9 0 1 0
Head 9 0 1 0

Updated by this run for fb2dae753cb772dbba8840fd9e9dbdafd2b0ce55.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for this test-only change.

No open issues 🎉

What changed in this PR

Adds characterization tests for #5541, documenting through assertions that schema-invalid records preserved by #4885 cannot be removed or repaired through the memory server’s existing tools. No production behavior changes.

Changes:

  • Adds eight protocol-level cases covering all nine tools through an SDK client.
  • Checks tool responses and exact file contents, including same-name entity creation and cascading deletion.
  • Uses isolated temporary files and suppresses expected error logs.
File Description
src/​memory/​__tests__/​invalid-record-removal.test.ts Pins invalid records’ invisibility to tools and preservation across writes.

Review effort: Balanced

@cliffhall

Copy link
Copy Markdown
Member Author

Copilot round 1: clean. The review reported no findings and no suppressed comments, so the loop ends here on the first clean round. Nothing was pushed after fb2dae75.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory: characterization test — a schema-invalid record cannot be removed by any tool

2 participants