Skip to content

fix(memory): give the real-endpoint HTTP tests a timeout clear of a slow runner - #5632

Open
cliffhall wants to merge 1 commit into
v2/mainfrom
v2/fix/5570-memory-http-tests-windows-timeout
Open

cliffhall wants to merge 1 commit into
v2/mainfrom
v2/fix/5570-memory-http-tests-windows-timeout

Conversation

@cliffhall

Copy link
Copy Markdown
Member

Closes #5570

Description

Gives memory's real-endpoint Streamable HTTP tests an explicit 30 s timeout, with the measurements behind it in a comment. Test-only: two describe lines and their comments.

  • src/memory/__tests__/http.test.ts: describe("Streamable HTTP", { timeout: REAL_ENDPOINT_TIMEOUT_MS }, …)
  • src/memory/__tests__/write-limits.test.ts: describe("write limits over Streamable HTTP", { timeout: 30_000 }, …)

The cause is a slow runner, not the endpoint, the client's negotiation or the file lock. The two tests are fast on Windows; they are the ones with the most sequential round trips, and each is the first real client connect in its file, so they are the first to cross Vitest's 5 s default when a runner slows everything down.

Server Details

  • Server: memory
  • Changes to: tests only

Motivation and Context

#5570: two tests timed out on Test memory (Windows) and passed on a rerun. The issue asks where the time goes and how often.

How often. Every typescript.yml run since the transport landed, from the job logs: 56 Windows memory legs ran these tests, and 1 failed (run 44246763473, attempt 1). Per file, on Windows:

File min median p90 max passing failed leg
http.test.ts (22 tests) 369 ms 568 ms 720 ms 822 ms 8751 ms
write-limits.test.ts (25 tests) 200 ms 316 ms 378 ms 460 ms 6648 ms

So they are not near 5 s in a normal run. Filesystem's and sequential-thinking's http.test.ts on Windows show no tail at all (max 414 ms and 1043 ms per file over 56 and 55 legs).

The failed leg was slow everywhere. Its suite took 34.1 s of test time against a 6.8 s median (the slowest passing leg: 12.1 s). Files that never open a socket were slowed as much as the HTTP ones:

File failed leg median passing
extra-fields.test.ts (in-memory transport) 5052 ms 269 ms
read-options.test.ts (in-memory transport) 4527 ms 354 ms
resource.test.ts 1184 ms 228 ms
atomic-save.test.ts 421 ms 73 ms

Where the time goes. A temporary diagnostic on this branch (not in this diff; commits 4ec6947c and c404e63e, dropped before the PR opened) timed each phase of the two tests and ran memory's whole suite repeatedly on Windows runners, with a 60 s test timeout so a slow test would finish and report:

  • Run 44375778932: 5 runners × 80 runs of the suite, with phase timings. Run 44383570850: 8 runners × 80, with every test's duration from Vitest's JSON report. 1,040 runs, 0 failures, and no test in the suite reached 1 s (slowest: 849 ms, the stdio smoke test).
  • The two tests (400 instrumented runs each): "serves a 2026-07-28 client" median 138 ms, p99 276 ms, max 673 ms; "holds the endpoint's write tools" median 72 ms, p99 210 ms, max 742 ms. Both maxima are the first run after install on one runner, where the first connect took 506 ms and 577 ms with the event loop blocked for about 250 ms of it.
  • Phases, medians (2026-07-28 case): start the endpoint 5 ms, connect 51 ms, tools/list 19 ms, create_entities 50 ms, open_nodes 6 ms. No retry, reconnection delay or fallback wait shows up anywhere; the pinned-era client and the probing default client behave the same.
  • The lock is ruled out. Over 1,600 acquisitions withFileLock got the lock on its first open every time (max 68 ms to acquire, 24 ms to release), so the Windows EPERM wait was never taken. saveGraph's write and rename took at most 20 ms and 11 ms.
  • Runners do slow down in bursts: on most of the 13 runners, several consecutive runs took 2 to 10 times the usual test time (up to 56 s of test time in one run), at the same wall-clock minute across runners. In those bursts no test reached 1 s. The failed leg is the far end of that: about five times the median overall.

That is why the change is a timeout rather than a change to the endpoint or the lock: there is no wait to remove. The limit exists to catch a hang, and 30 s still does; it is about 40 times the slowest these tests have been measured.

How Has This Been Tested?

No client-observable surface, so a targeted probe:

  • The timeouts take effect: with both set to 1 locally, all 7 cases in Streamable HTTP and the one in write limits over Streamable HTTP fail with Test timed out in 1ms, and nothing else does.
  • npm test -w src/memory: 297 passed. npm run local:gate: exit 0.
  • Test memory (Windows) on this PR's commit, repeated: see the comment below for each run and its durations.

Breaking Changes

None.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Protocol Documentation (not applicable: no protocol behavior changes)
  • My changes follow MCP security best practices (not applicable: tests only)
  • I have updated the server's README accordingly (not applicable: nothing a user sees changes)
  • I have added a changeset (npm run changeset) if this changes what a TypeScript server publishes (not applicable: tests are not published)
  • I have tested this with an LLM client (not applicable: no server-facing change)
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling (not applicable: tests only)
  • I have documented all environment variables and configuration options (not applicable: none added)

Additional context

The timeout is on the two describes rather than on the two tests that failed: every case in them drives a real endpoint, and in the failed leg the listener case beside them took 1540 ms, against under 300 ms in every passing leg. A suite-wide testTimeout was not used, since the in-memory tests have far more headroom (the slowest took 1420 ms in the failed leg).

🤖 Generated with Claude Code

…low runner

Two Streamable HTTP tests timed out at Vitest's 5 s default on one Windows
CI leg in 56. Measured over 1,040 runs of the suite on 13 Windows runners,
they take a median of 140 ms and 72 ms and never reached 1 s; the lock was
always taken on the first try and the save's rename took at most 11 ms. In
the failed leg the whole suite ran five times slower than its median, with
in-memory tests slowed as much as the HTTP ones, so the cause is the
runner, and the tests with the most round trips crossed the limit.

Both describes now carry an explicit 30 s timeout with the measurements.

Closes #5570

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall cliffhall added the v2 label Nov 18, 2026
@changeset-bot

changeset-bot Bot commented Nov 18, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 08f87f0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cliffhall
cliffhall requested a balanced review from Copilot November 18, 2026 20:34
@github-actions

Copy link
Copy Markdown
Contributor

memory: MCP interface diff

Base: 7f383a8e0f34 (Merge pull request #5597 from modelcontextprotocol/changeset-release/v2/main)
Head: the build in this checkout, at 4eaf1b6c4a0e (Merge 08f87f0 into 7f383a8)

Diffed because it was named (--server).

✅ No interface changes detected.

Tools Prompts Resources Resource templates
Base 9 0 1 0
Head 9 0 1 0

Updated by this run for 08f87f0cad8241cd4d2d968bf19d0c7ce7c3118f.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the HTTP test-timeout changes intended to accommodate slow CI runners.

No open issues 🎉

What changed in this PR

Addresses #5570 by giving the memory server’s real-endpoint HTTP tests more time on slow CI runners, without changing server behavior or adding retries.

Changes:

  • Sets a 30-second timeout on both Streamable HTTP test groups.
  • Adds comments explaining the timeout using measured Windows CI durations and runner slowdowns.
  • Keeps the default timeout for other tests.
File Description
src/​memory/​__tests__/​write-limits.test.ts Adds a 30-second timeout and measurement context to the HTTP write-limit test group.
src/​memory/​__tests__/​http.test.ts Defines and applies a 30-second timeout to the HTTP test group, with supporting CI measurements.

Review effort: Balanced

@cliffhall

Copy link
Copy Markdown
Member Author

Test memory (Windows), repeated on this PR's commit 08f87f0c: run 44392432438, the leg re-run five times on the unchanged commit, six passes in six. This is not a retry of a failure: each attempt passed, and they are here only as the repeated-pass record #5570 asks for.

Attempt http.test.ts write-limits.test.ts Suite test time Wall
1 622 ms 350 ms 7.58 s 6.26 s
2 733 ms 373 ms 7.83 s 6.46 s
3 491 ms 250 ms 6.14 s 4.87 s
4 638 ms 371 ms 7.78 s 6.34 s
5 659 ms 384 ms 7.88 s 6.42 s
6 547 ms 330 ms 7.01 s 5.25 s

Six passes say little about a failure seen once in 56 legs; the weight is in the 1,040 diagnostic runs in the description (runs 44375778932 and 44383570850), where no test in the suite reached 1 s.

Copilot round 1: clean, no findings ("Approval recommended", no open issues). The loop ends here on the first clean round; nothing was pushed after it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory: two Streamable HTTP tests time out intermittently on Windows CI

2 participants