From 3ef405f2f4c0d5e0ff4446c37520177022518454 Mon Sep 17 00:00:00 2001 From: Andy Wang Date: Wed, 18 Nov 2026 21:07:46 +0800 Subject: [PATCH] fix: cancel in-flight requests when their session ends ## Motivation and Context `StreamableHTTPTransport` removes a stateful session when the client terminates it with `DELETE`, when it passes `session_idle_timeout`, when its GET stream breaks without an event store, and when the transport is closed. Requests still running in that session were left running: their `MCP::Cancellation` was never flipped, so `server_context.cancelled?` stayed false, `on_cancel` hooks registered on `server_context.cancellation` never ran, and a long tool call kept working for a client that had gone. In JSON response mode its result was still written to the POST of a session that no longer existed. Only nested server-to-client waits were released, through the `:session_closed` that `cleanup_session_unsafe` pushes to them. Each removal path now releases the session through one `release_session`, which closes its streams as before and then cancels its in-flight requests with `ServerSession#cancel_all_incoming` and the reason `"Session terminated"`. Handlers observe it as they would `notifications/cancelled`, and their responses are suppressed the same way. The cancellations run once `@mutex` is released, as the stream closes already do: `Cancellation#cancel` fires its hooks synchronously, and a hook may call back into the transport, whose lock is not reentrant. The TypeScript SDK does the same in `Protocol#_onclose`, which aborts the handlers of every in-flight request when the transport closes. ## How Has This Been Tested? New tests in `test/mcp/server/transports/streamable_http_transport_test.rb` hold a tool call in flight with `Queue` barriers. They check that a `DELETE` cancels the request with the reason `"Session terminated"` and its POST is answered 202 with no body; that terminating one session leaves the requests of another running to their result; that `close`, the reaper, a request, a notification, a targeted notification, a broadcast, an `initialize` reclaiming an expired slot, and a failed GET stream write each cancel the requests of the session they remove; and that the hooks run without the transport lock held. All of them fail against the previous library, and skipping the cancellation on any one removal path fails the matching case. ## Breaking Changes A request still running when its session is removed no longer delivers its result. In JSON response mode its POST is answered 202 with no body, as for a request cancelled with `notifications/cancelled`. Over SSE the termination already closed the POST stream, so the result was lost before as well. This includes a request that outlives `session_idle_timeout` (30 minutes by default), since a session's idle time is counted from the start of its latest request. --- docs/_server/cancellation.md | 9 + .../transports/streamable_http_transport.rb | 79 ++++---- lib/mcp/server_session.rb | 8 + .../streamable_http_transport_test.rb | 171 ++++++++++++++++++ 4 files changed, 228 insertions(+), 39 deletions(-) diff --git a/docs/_server/cancellation.md b/docs/_server/cancellation.md index 1939a24e..5c378655 100644 --- a/docs/_server/cancellation.md +++ b/docs/_server/cancellation.md @@ -104,6 +104,15 @@ end Handlers that do not declare a `server_context:` keyword continue to work unchanged - the opt-in detection only wraps the context when the block signature asks for it. +## Requests of a Terminated Session Are Cancelled + +When `StreamableHTTPTransport` removes a stateful session while requests are still running in it, those requests +are cancelled as if the client had sent `notifications/cancelled` for each of them. This happens when the client terminates +the session with `DELETE`, when the session passes `session_idle_timeout`, when its GET stream breaks without an event store +to resume from, and when the transport is closed. `server_context.cancelled?` turns true, `on_cancel` hooks registered on +`server_context.cancellation` run with the reason `"Session terminated"`, and the response is suppressed, since no client +is left to receive it. + ## Nested Server-to-Client Requests Are Cancelled Automatically When a tool handler is waiting on a nested server-to-client request diff --git a/lib/mcp/server/transports/streamable_http_transport.rb b/lib/mcp/server/transports/streamable_http_transport.rb index cc862774..eb2d3375 100644 --- a/lib/mcp/server/transports/streamable_http_transport.rb +++ b/lib/mcp/server/transports/streamable_http_transport.rb @@ -431,6 +431,9 @@ class StreamWriteTimeoutError < IOError; end # How long `close` waits in all for the closing results of the open `subscriptions/listen` streams # before closing the streams still open without one. LISTEN_TEARDOWN_TIMEOUT = 5 + # The reason the requests still in flight in a session are cancelled with when the session goes away: terminated by + # the client's DELETE, expired, torn down with its broken GET stream, or removed by `close`. + SESSION_TERMINATED_REASON = "Session terminated" # Loopback hosts always accepted by DNS rebinding protection. A locally bound MCP server (the canonical pattern) is # protected out of the box; non-loopback deployments widen the list via `allowed_hosts:`. @@ -570,10 +573,7 @@ def close @sessions.each_key.filter_map { |session_id| cleanup_session_unsafe(session_id) } end - removed_sessions.each do |session| - close_stream_safely(session[:get_sse_stream]) - close_post_request_streams(session) - end + release_sessions(removed_sessions) end # Sessions a server-wide log broadcast is offered to, each applying its own level or the server default. @@ -642,26 +642,26 @@ def deliver_targeted_notification(notification, session_id, related_request_id) # (e.g. `resources/updated`, `elicitation/complete`) still flow via GET SSE. return false if @enable_json_response && related_request_id - streams_to_close = [] + removed_sessions = [] live = @mutex.synchronize do next false unless (session = @sessions[session_id]) if session_expired?(session) - cleanup_and_collect_stream(session_id, streams_to_close) + cleanup_and_collect_session(session_id, removed_sessions) next false end true end - close_streams(streams_to_close) + release_sessions(removed_sessions) return false unless live store_and_write(session_id, related_request_id, notification.to_json) end def deliver_broadcast_notification(notification) - streams_to_close = [] + removed_sessions = [] # Snapshot the sessions under the lock, then write to each outside it. A session whose GET stream is # closed at the moment still takes the notification when it has opened that stream before and an event @@ -681,13 +681,13 @@ def deliver_broadcast_notification(notification) end expired_session_ids.each do |session_id| - cleanup_and_collect_stream(session_id, streams_to_close) + cleanup_and_collect_session(session_id, removed_sessions) end collected end - close_streams(streams_to_close) + release_sessions(removed_sessions) json = notification.to_json targets.count do |session_id| @@ -707,6 +707,7 @@ def deliver_broadcast_notification(notification) # from its last event id (SEP-1699), and the events it misses meanwhile are stored for it. def drop_broken_stream(session_id, stream, related_request_id) streams_to_close = [] + removed_sessions = [] @mutex.synchronize do session = @sessions[session_id] @@ -722,12 +723,13 @@ def drop_broken_stream(session_id, stream, related_request_id) if @event_store streams_to_close << detach_get_stream(session) else - cleanup_and_collect_stream(session_id, streams_to_close) + cleanup_and_collect_session(session_id, removed_sessions) end end end close_streams(streams_to_close) + release_sessions(removed_sessions) end def close_streams(streams) @@ -945,10 +947,7 @@ def reap_expired_sessions end end - removed_sessions.each do |session| - close_stream_safely(session[:get_sse_stream]) - close_post_request_streams(session) - end + release_sessions(removed_sessions) end # One `write` per frame, so the frames of concurrent writers cannot interleave in the middle of each other. @@ -2020,24 +2019,21 @@ def handle_delete(request) success_response end - # Removes the session, closes its streams outside the lock, and returns the removed session, or nil when + # Removes the session, releases it outside the lock, and returns the removed session, or nil when # there was none to remove. def cleanup_session(session_id) session = @mutex.synchronize do cleanup_session_unsafe(session_id) end - if session - close_stream_safely(session[:get_sse_stream]) - close_post_request_streams(session) - end + release_session(session) if session session end - # Removes a session from `@sessions` and returns it. Does not close the stream. - # Callers must close the stream outside the mutex to avoid holding the lock during - # potentially blocking I/O. + # Removes a session from `@sessions` and returns it. Does not close the stream or cancel the in-flight requests. + # Callers must `release_session` it outside the mutex to avoid holding the lock during + # potentially blocking I/O and the requests' cancellation hooks. def cleanup_session_unsafe(session_id) session = @sessions.delete(session_id) @@ -2051,11 +2047,25 @@ def cleanup_session_unsafe(session_id) session end - def cleanup_and_collect_stream(session_id, streams_to_close) + # Removes a session under `@mutex` and adds it to `removed_sessions`, for the caller to release once outside the lock. + def cleanup_and_collect_session(session_id, removed_sessions) return unless (removed = cleanup_session_unsafe(session_id)) - streams_to_close << removed[:get_sse_stream] - removed[:post_request_streams]&.each_value { |stream| streams_to_close << stream } + removed_sessions << removed + end + + # Ends what a removed session still has running: closes its streams and cancels its in-flight requests, whose handlers + # observe the termination through `server_context.cancelled?` and whose responses are suppressed as for + # `notifications/cancelled`. Runs outside `@mutex`: a close can block, and a cancellation fires the request's + # `on_cancel` hooks synchronously, which may call back into the transport. + def release_session(session) + close_stream_safely(session[:get_sse_stream]) + close_post_request_streams(session) + session[:server_session]&.cancel_all_incoming(reason: SESSION_TERMINATED_REASON) + end + + def release_sessions(sessions) + sessions.each { |session| release_session(session) } end # A close can block on the same full buffers a write does (a TLS stream writes its closing alert, a buffered one flushes), @@ -2371,7 +2381,7 @@ def handle_initialization(request, body_string, body) # so the cap rejects only when genuinely full rather than up to a reaper interval after sessions expired. if @max_sessions && @sessions.size >= @max_sessions @sessions.each_key.select { |id| session_expired?(@sessions[id]) }.each do |id| - cleanup_and_collect_stream(id, reclaimed) + cleanup_and_collect_session(id, reclaimed) end end @@ -2400,7 +2410,7 @@ def handle_initialization(request, body_string, body) true end - reclaimed.each { |stream| close_stream_safely(stream) } + release_sessions(reclaimed) return too_many_sessions_response unless inserted end @@ -2596,13 +2606,7 @@ def validate_and_touch_session(session_id) nil end - if removed - close_stream_safely(removed[:get_sse_stream]) - - removed[:post_request_streams]&.each_value do |stream| - close_stream_safely(stream) - end - end + release_session(removed) if removed response end @@ -2657,10 +2661,7 @@ def session_active?(session_id) true end - if removed - close_stream_safely(removed[:get_sse_stream]) - close_post_request_streams(removed) - end + release_session(removed) if removed active end diff --git a/lib/mcp/server_session.rb b/lib/mcp/server_session.rb index 7b0f0110..a07429e8 100644 --- a/lib/mcp/server_session.rb +++ b/lib/mcp/server_session.rb @@ -106,6 +106,14 @@ def cancel_incoming(request_id:, reason: nil) cancellation&.cancel(reason: reason) end + # Flips the `Cancellation` of every request still in flight, for a transport ending the session they belong to: + # their handlers observe it as they would `notifications/cancelled`, and their responses are suppressed the same way. + # The cancellations run outside the registry lock, since each fires its `on_cancel` hooks synchronously. + def cancel_all_incoming(reason: nil) + cancellations = @in_flight_mutex.synchronize { @in_flight.values } + cancellations.each { |cancellation| cancellation.cancel(reason: reason) } + end + # Sends `notifications/cancelled` to the peer for a previously-issued request. # Also unblocks any transport-level `send_request` waiting on a response for `request_id`. def cancel_request(request_id:, reason: nil) diff --git a/test/mcp/server/transports/streamable_http_transport_test.rb b/test/mcp/server/transports/streamable_http_transport_test.rb index fc5b8c49..cd8145ed 100644 --- a/test/mcp/server/transports/streamable_http_transport_test.rb +++ b/test/mcp/server/transports/streamable_http_transport_test.rb @@ -4026,6 +4026,155 @@ def string transport.close end + test "terminating a session with DELETE cancels its in-flight requests and suppresses their responses" do + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true) + server.transport = transport + cancellations = [] + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: cancellations, started: started, release: release) + session_id, = start_session(transport) + + call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + started.pop + + delete_response = transport.handle_request( + create_rack_request("DELETE", "/", { "HTTP_MCP_SESSION_ID" => session_id }), + ) + + assert_equal(200, delete_response[0]) + assert_equal([["call-1", "Session terminated"]], cancellations) + + release << true + assert_equal(202, call.value[0]) + ensure + release&.push(true) + call&.join + transport&.close + end + + test "terminating a session cancels only the requests in flight in that session" do + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true) + server.transport = transport + cancellations = [] + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: cancellations, started: started, release: release) + terminated_session_id, = start_session(transport) + surviving_session_id, = start_session(transport) + + terminated_call = Thread.new { transport.handle_request(held_tool_call(terminated_session_id, "call-1")) } + surviving_call = Thread.new { transport.handle_request(held_tool_call(surviving_session_id, "call-2")) } + 2.times { started.pop } + + transport.handle_request(create_rack_request("DELETE", "/", { "HTTP_MCP_SESSION_ID" => terminated_session_id })) + + assert_equal([["call-1", "Session terminated"]], cancellations) + + 2.times { release << true } + response = surviving_call.value + assert_equal(200, response[0]) + assert_equal("done", JSON.parse(response[2].first).dig("result", "content", 0, "text")) + ensure + 2.times { release&.push(true) } + terminated_call&.join + surviving_call&.join + transport&.close + end + + test "every other path that removes a session cancels the requests in flight in it" do + expire = ->(transport, session_id) { transport.instance_variable_get(:@sessions)[session_id][:last_active_at] -= 61 } + session_post = ->(session_id, body) do + create_rack_request("POST", "/", { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, body.to_json) + end + attach_get_stream = ->(transport, session_id) do + TestStream.new.tap { |stream| transport.instance_variable_get(:@sessions)[session_id][:get_sse_stream] = stream } + end + + removals = { + "closing the transport" => ->(transport, _session_id) { transport.close }, + "the reaper finding it expired" => ->(transport, session_id) do + expire.call(transport, session_id) + transport.send(:reap_expired_sessions) + end, + "a request finding it expired" => ->(transport, session_id) do + expire.call(transport, session_id) + transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", id: "ping-1", method: "ping" })) + end, + "a notification finding it expired" => ->(transport, session_id) do + expire.call(transport, session_id) + transport.handle_request(session_post.call(session_id, { jsonrpc: "2.0", method: "notifications/initialized" })) + end, + "a notification sent to it after it expired" => ->(transport, session_id) do + expire.call(transport, session_id) + transport.send_notification("notifications/message", { level: "info", data: "late" }, session_id: session_id) + end, + "a broadcast finding it expired" => ->(transport, session_id) do + attach_get_stream.call(transport, session_id) + expire.call(transport, session_id) + transport.send_notification(Methods::NOTIFICATIONS_TOOLS_LIST_CHANGED) + end, + "an initialize reclaiming its expired slot" => ->(transport, session_id) do + expire.call(transport, session_id) + start_session(transport) + end, + "a failed write to its GET stream" => ->(transport, session_id) do + transport.send(:drop_broken_stream, session_id, attach_get_stream.call(transport, session_id), nil) + end, + } + + removals.each do |removal, remove_session| + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true, session_idle_timeout: 60, max_sessions: 1) + server.transport = transport + cancellations = [] + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: cancellations, started: started, release: release) + session_id, = start_session(transport) + + call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + started.pop + + remove_session.call(transport, session_id) + + assert_equal([["call-1", "Session terminated"]], cancellations, "session removed by #{removal}") + ensure + release&.push(true) + call&.join + transport&.close + end + end + + test "cancellation hooks of a terminated session's requests run outside the transport lock" do + server = Server.new(name: "test", tools: [], prompts: [], resources: []) + transport = StreamableHTTPTransport.new(server, enable_json_response: true) + server.transport = transport + started = Queue.new + release = Queue.new + define_held_tool(server, cancellations: [], started: started, release: release) + session_id, server_session = start_session(transport) + + call = Thread.new { transport.handle_request(held_tool_call(session_id, "call-1")) } + started.pop + + # A hook may call back into the transport (to log, or to tell the peer about a nested request), + # which would raise on the non-reentrant lock if the hook ran while the session was being removed under it. + mutex = transport.instance_variable_get(:@mutex) + lock_held = [] + server_session.lookup_in_flight("call-1").on_cancel { lock_held << mutex.owned? } + + transport.handle_request(create_rack_request("DELETE", "/", { "HTTP_MCP_SESSION_ID" => session_id })) + + assert_equal([false], lock_held) + ensure + release&.push(true) + call&.join + transport&.close + end + test "send_request raises on error response from client" do # Create session. init_request = create_rack_request( @@ -8639,6 +8788,28 @@ def start_session(transport) [session_id, transport.instance_variable_get(:@sessions)[session_id][:server_session]] end + # Defines `held_tool`, which records each cancellation of its request as `[request id, reason]`, pushes to `started` + # once running, and returns only after something is pushed to `release`, so a test can end the session mid-call. + def define_held_tool(server, cancellations:, started:, release:) + server.define_tool(name: "held_tool") do |server_context:| + cancellation = server_context.cancellation + cancellation.on_cancel { |reason| cancellations << [cancellation.request_id, reason] } + started << true + release.pop + + Tool::Response.new([{ type: "text", text: "done" }]) + end + end + + def held_tool_call(session_id, request_id) + create_rack_request( + "POST", + "/", + { "CONTENT_TYPE" => "application/json", "HTTP_MCP_SESSION_ID" => session_id }, + { jsonrpc: "2.0", id: request_id, method: "tools/call", params: { name: "held_tool", arguments: {} } }.to_json, + ) + end + # A `tools/call` for `victim_tool` under a caller-chosen request id, with a progress token so # the tool's `report_progress` has somewhere to go. def colliding_tool_call(session_id, request_id)