feat: Add experimental async transport (port of PR #4572)#5646
Draft
feat: Add experimental async transport (port of PR #4572)#5646
Conversation
Contributor
Semver Impact of This PR🟡 Minor (new features) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨Anthropic
Pydantic Ai
Other
Bug Fixes 🐛
Documentation 📚
Internal Changes 🔧Anthropic
Docs
Openai Agents
Other
🤖 This preview updates automatically when you update the PR. |
Contributor
Codecov Results 📊✅ 31 passed | Total: 31 | Pass Rate: 100% | Execution Time: 12.26s 📊 Comparison with Base Branch
All tests are passing successfully. ❌ Patch coverage is 20.32%. Project has 15187 uncovered lines. Files with missing lines (7)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 26.19% 27.15% +0.96%
==========================================
Files 189 189 —
Lines 20485 20847 +362
Branches 6698 6806 +108
==========================================
+ Hits 5364 5660 +296
- Misses 15121 15187 +66
- Partials 463 498 +35Generated by Codecov Action |
Contributor
Codecov Results 📊Generated by Codecov Action |
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Add an experimental async transport using httpcore's async backend,
enabled via `_experiments={"transport_async": True}`.
This is a manual port of PR #4572 (originally merged into `potel-base`)
onto the current `master` branch.
Key changes:
- Refactor `BaseHttpTransport` into `HttpTransportCore` (shared base) +
`BaseHttpTransport` (sync) + `AsyncHttpTransport` (async, conditional
on httpcore[asyncio])
- Add `Worker` ABC and `AsyncWorker` using asyncio.Queue/Task
- Add `close_async()` / `flush_async()` to client and public API
- Patch `loop.close` in asyncio integration to flush before shutdown
- Add `is_internal_task()` ContextVar to skip wrapping Sentry-internal tasks
- Add `asyncio` extras_require (`httpcore[asyncio]==1.*`)
- Widen anyio constraint to `>=3,<5` for httpx and FastAPI
Refs: GH-4568
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BYK
force-pushed
the
feat/async-transport
branch
from
8c808bf to
4f8a00c
Compare
The base class _make_pool returns a union of sync and async pool types, so mypy sees _pool.request() as possibly returning a non-awaitable. Add type: ignore[misc] since within AsyncHttpTransport the pool is always an async type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The asyncio extra on httpcore pulls in anyio, which conflicts with starlette's anyio<4.0.0 pin and causes pip to downgrade httpcore to 0.18.0. That old version crashes on Python 3.14 due to typing.Union not having __module__. Keep httpcore[http2] in requirements-testing.txt (shared by all envs) and add httpcore[asyncio] only to linters, mypy, and common envs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- AsyncWorker.kill() now calls self._task.cancel() before clearing the reference, preventing duplicate consumers if submit() is called later - close() with AsyncHttpTransport now does best-effort sync cleanup (kill transport, close components) instead of silently returning - flush()/close() log warnings instead of debug when async transport used - Add __aenter__/__aexit__ to _Client for 'async with' support Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Asyncio and gevent don't mix — async tests using asyncio.run() fail under gevent's monkey-patching. Add skip_under_gevent decorator to all async tests in test_transport.py and test_client.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Python 3.6 doesn't support PEP 563 (from __future__ import annotations). Use string-quoted annotations instead, matching the convention used in the rest of the SDK. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 77 tests covering: - AsyncWorker lifecycle (init, start, kill, submit, flush, is_alive) - AsyncWorker edge cases (no loop, queue full, cancelled tasks, pid mismatch) - HttpTransportCore methods (_handle_request_error, _handle_response, _update_headers, _prepare_envelope) - make_transport() async detection (with/without loop, integration, http2) - AsyncHttpTransport specifics (header parsing, capture_envelope, kill) - Client async methods (close_async, flush_async, __aenter__/__aexit__) - Client component helpers (_close_components, _flush_components) - asyncio integration (patch_loop_close, _create_task_with_factory) - ContextVar utilities (is_internal_task, mark_sentry_task_internal) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use a sync test to test the no-running-loop path — there's genuinely no running loop in a sync test, so no mock needed and no leaked coroutines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After AsyncWorker.kill() cancels tasks, the event loop needs a tick to actually process the cancellations. Without this, pytest reports PytestUnraisableExceptionWarning for never-awaited coroutines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When kill() cancels the _target task while it's waiting on queue.get(), the CancelledError propagates through the coroutine. Without catching it, the coroutine gets garbage collected with an unhandled exception, causing pytest's PytestUnraisableExceptionWarning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On Python 3.8, cancelled asyncio coroutines that were awaiting Queue.get() raise GeneratorExit during garbage collection, triggering PytestUnraisableExceptionWarning. This is a Python 3.8 asyncio limitation, not a real bug. Suppress the warning for async worker tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment on lines
+1156
to
+1164
| try: | ||
| asyncio.get_running_loop() | ||
| if async_integration: | ||
| if use_http2_transport: | ||
| logger.warning( | ||
| "HTTP/2 transport is not supported with async transport. " | ||
| "Ignoring transport_http2 experiment." | ||
| ) | ||
| transport_cls = AsyncHttpTransport |
There was a problem hiding this comment.
Race condition in async transport selection: event loop may not persist after check
The code calls asyncio.get_running_loop() to verify an event loop exists before setting transport_cls = AsyncHttpTransport, but by the time the transport is actually instantiated at line 1193, the loop could be gone or different. The AsyncHttpTransport.__init__ also calls asyncio.get_running_loop() and stores it, but if the context changes between the check in make_transport and instantiation, this could lead to inconsistent state. However, in typical usage patterns, this is unlikely to manifest because both happen in the same synchronous call stack.
Verification
Verified by reading transport.py lines 764-768 where AsyncHttpTransport.init calls asyncio.get_running_loop() and stores it in self.loop. The check at line 1157 and the instantiation at line 1193 both happen synchronously, so unless there's an unusual threading scenario, this is low-risk. The AsyncWorker.start() method at worker.py:231-245 also calls get_running_loop() safely.
Identified by Warden find-bugs · S6B-5MM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an experimental async transport using httpcore's async backend,
enabled via
_experiments={"transport_async": True}.This is a manual port of PR #4572 (originally merged into
potel-base)onto the current
masterbranch.Key changes
transport.py: Refactor
BaseHttpTransportintoHttpTransportCore(shared base) +
BaseHttpTransport(sync) +AsyncHttpTransport(async, conditional on
httpcore[asyncio]). Extract shared helpers:_handle_request_error,_handle_response,_update_headers,_prepare_envelope. Updatemake_transport()to detect thetransport_asyncexperiment.worker.py: Add
WorkerABC base class andAsyncWorkerimplementation using
asyncio.Queue/asyncio.Task.client.py: Add
close_async()/flush_async()with async-vs-synctransport detection. Extract
_close_components()/_flush_components().api.py: Expose
flush_async()as a public API.integrations/asyncio.py: Patch
loop.closeto flush pending eventsbefore shutdown. Skip span wrapping for internal Sentry tasks.
utils.py: Add
is_internal_task()/mark_sentry_task_internal()via ContextVar for async task filtering.
setup.py: Add
"asyncio"extras_require (httpcore[asyncio]==1.*).config.py / tox.ini: Widen anyio to
>=3,<5for httpx and FastAPI.Notes
tox.iniwas manually edited (the generation script requires afree-threaded Python interpreter). A full regeneration should be done
before merge.
Refs: GH-4568