Repository navigation
feat(rtc)!: move the WebRTC transport from aiortc to the Rust Video SDK - #303
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 Walkthrough
Merge Risk: 🟡 Moderate · up to
|
8805bd0 to
a17f9eb
Compare
Build - rust-toolchain.toml pins Rust 1.88, the minimum version of the Rust Video SDK. - Cargo.toml builds getstream._native from rustsrc/ against the SDK (stream-video-rust, branch fix/compatibility-fixes). - pyproject.toml uses maturin instead of hatchling, and uv cache keys make uv sync rebuild after Rust changes. - release-please also updates the version in Cargo.toml and Cargo.lock. - AGENTS.md lists the native build requirements. Binding - Client/Call: join, leave, session_id and calling_state as awaitables and properties; SDK errors raise Error and RtcError. join() waits until a cancelled join has finished its cleanup. - events() and tracks(): async iterators that end when the call ends; participants() and call_state(). - Media: RemoteTrack.next_pcm()/next_video_frame() return numpy arrays without a copy; LocalAudioTrack/LocalVideoTrack, publish_audio(), publish_video() and update_subscriptions(). Logging - SDK tracing events and third-party log records reach Python logging through a bounded queue and one forwarding thread, so a thread that logs never waits for the GIL. - configure_logging(logger, level, third_party_level=WARNING); records keep the time of the Rust event. - Client(log_bodies=True) logs redacted request and response bodies.
SDK logs are now visible in pytest runs
Fewer requests get sent to the backend this way
- Call.stop_publish_audio, stop_publish_video, stop_publish_screen_share - Call.publish_screen_share (VP8) - LocalAudioTrack(pcm_queue_capacity) in seconds - __repr__ for participants, events, remote tracks and frames - AGENTS.md: sdk_logs is for assertions on records
- Emit SFU events, coordinator custom events, connection state changes, call_ended, track_added and audio from the SDK event and track streams, with stream-py payload classes instead of protobuf. - Join with an API secret or a user token; apply subscription configs. - Publish aiortc audio and video tracks through AudioForwarder and VideoForwarder (VP9 video, mono audio, unpaced AudioStreamTrack option). - Key participants_state by session_id, include the local participant and refill it after lost events. - Add RemoteTrack.video_frames(): an independent latest-frame stream for each reader, in place of next_video_frame(). - Remove local recording. - Add migration.md with the breaking changes versus 4.1.0.
Consumers import RemoteTrack, RemoteParticipant, TrackType, VideoFrameStream, VideoFrame and the emitted event classes from the public package instead of getstream._rust.
- A new video track reuses the stopped video sender; test added. - Set the extension version to 6.1.1, as in pyproject.toml. - Document the 20 ms minimum PCM queue and video republishing.
- add_tracks unpublishes the earlier audio or video track before it publishes a new one, so a new track no longer fails with MediaError when the earlier one is live or not yet unpublished. - Bump the Rust SDK revision to 147e04a.
- Delete the old join, signaling, reconnection and peer connection code and the attributes and methods that only it used. - rtc.join takes subscription_config explicitly; max_join_retries, drain_video_frames and other keyword arguments are removed.
- Delete the peer connection, signaling, coordinator WebSocket, reconnection, stats, tracer, location discovery and aiortc encoder patch modules with their tests. - Keep only the subscription config dataclasses in tracks.py and the audio helpers in track_util.py.
scipy, soundfile, websockets, websocket-client, structlog, tenacity and ping3 were used only by the deleted aiortc transport code.
A remote track that is dropped during a mute is delivered again as a new track_added when its media flows after the republish.
It plays back the browser participant's audio and shows their camera mirrored, through rtc.join, the audio and track_added events, RemoteTrack.video_frames() and add_tracks.
When the call ends without leave(), the SDK stops the published tracks. The forwarders kept writing, and the publish task ended with an IllegalStateError that nothing retrieved.
It writes the audio event of each participant session to its own WAV file.
- Cast the aiortc frames and their arrays to the types that the forwarders write. - Narrow the coordinator and client events by their class. - Require user_id in rtc.join and ConnectionManager, and raise TypeError for a call without an id.
The SDK tasks keep a joined call alive, so a Call that Python freed without leave() stayed in the call until the process exited. Its Drop now starts leave() on the runtime; it takes no SDK lock with the GIL held.
The SDK also rejects a call_event_capacity above 65536.
The tracks of a call stay in its queue after the call ends, so after a new join of the same Call, a new TrackStream first gave the dead tracks of the earlier join. join() now drops them before the SDK join, in Idle or Left only. The drop does not wait for the queue lock: a stream that holds it waits for a track, so the queue is empty, and ConnectionManager reads tracks before it joins. The TrackStream stub also documents the RtcError after a queue overflow.
The failure path of connect() cancels the event tasks, so nothing set the stop event, and wait() did not return. It is now set before the leave, so a cancellation during the leave cannot block wait() either.
The check for aiortc came after the imports of getstream_rtc and of modules that import aiortc, so a missing webrtc extra gave "No module named 'getstream_rtc'" instead of the pip install hint. The check now covers both imports and comes first.
The four WAV files over 256 KB in tests/assets had no user, and scripts/create_test_assets.py only made two of them. test_connection_manager loaded .env at import although the session fixture load_env already does, test_frame_timing requested an unused monkeypatch, and the custom event test was outside a class with an unannotated fixture.
test_cancelled_join_allows_new_join cancelled after a fixed 0.1 s, so a join that completed faster made pytest.raises(CancelledError) fail. The test now cancels when client_events() gives JOINING, while the join is in progress.
The members compare equal to their int values but had no hash, so they could not be dict keys or set members, and set(participant.published_tracks) raised TypeError, unlike the 6.1.1 values. __hash__ now returns the int value, which keeps the hash consistent with that equality. The stub declares plain classes instead of Enum, so type checkers reject name, value and iteration, which do not exist at runtime.
Only scripts/create_test_assets.py used them, and it is deleted.
When the source of a published track ended, its forwarding task awaited the SDK stop in finally. An add_tracks() at that moment cancelled the task, and the cancel dropped the SDK stop: the old track kept the only publish option of its kind, so the new publish and every later one raised MediaError until a reconnect or a leave. The task now shields the stop and, on a cancel, waits for it before it raises.
The LocalAudioTrack capacity docstring now names the ValueError for a negative, infinite or NaN value. The audio forwarder comment no longer says that aiortc's Opus encoder accepts only s16, two ConnectionManager comments no longer refer to the aiortc version, and the video forwarder explains that duration is an estimate from the previous gap.
test_no_records_after_logging_is_stopped could not fail: the autouse fixture forwards at INFO, so the DEBUG overflow record was filtered also with forwarding on. It now uses sdk_logs. Two event loops passed with no assertion when the stream ended early; they now fail. The cancelled writer tasks are awaited, so a writer error shows in the result, and the receivers that waited without a timeout now time out.
The test functions of the rtc test files had no return type, and the client fixture of test_connection_manager had none either.
A live check showed that on a kick or a block the SFU sends call_ended (reason KICKED) and an error with code 403 and no reconnect strategy, not DISCONNECT, and that the SDK leaves because of the call_ended. With block=True the state passes RECONNECTING before LEFT. The "Not verified" section is removed: the event names match the SDK, and no path waits for the GIL while it holds an SDK lock.
No test used mia.mp3, mia.json, formant_speech_16k.wav, speech_tone.wav or test_tone_16k.wav. mia.mp3 and mia.json were only the input of the deleted scripts/create_test_assets.py.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0c49760. Configure here.
Python 3.10 to 3.12 call Handler.release() in Handler.handle(). The Event attribute of the same name made each record raise TypeError, so caplog did not get the records.
The SDK replaced LocalVideoTrack::write_i420, which took the length of a frame, with write_i420_at, which takes its pts. The binding follows as LocalVideoTrack.write_i420_at(data, width, height, pts). The SDK set the RTP timestamp of the next frame from the length, and the forwarder could only give the gap to the previous frame, so after a change of the frame rate each timestamp was one frame late. The forwarder now gives each frame the sum of the gaps to it, so the RTP timestamps keep the spacing of the source times; a frame without a later time than the previous one still comes 1/30 s after it, because the SDK drops a frame whose pts does not increase.
Its last item, the loss of an item when a pending read is cancelled, affects only code that reads the getstream_rtc streams directly. The docstrings of EventStream, TrackStream and RemoteTrack.next_pcm describe it.
The crate has the same code as 0.2.0-preview.1; only the version changes.
The name matches MIGRATION_v2_to_v3.md. The guide gives the install command for 7.x and the getstream-rtc requirement, and states that a cancelled next_pcm() can lose a frame.
Each SHA is the commit that the earlier major tag pointed to; the comment gives its full version.
aiortc supports TURN over TCP and TLS, but it uses only the first TURN URL, so the UDP-only TURN of the SDK is not a change compared with 6.1.1.

Summary
getstream.video.rtcno longer implements the WebRTC transport with aiortc. TheRust Video SDK (stream-video-rust)
now does the coordinator join, the SFU selection, the SFU signaling, the peer
connections, the retries and the reconnects. The new package
getstream-rtc(import name
getstream_rtc) wraps the SDK (crategetstream0.2.0) with PyO3.rtc.join()andConnectionManagerstay the public API.ConnectionManagerisnow a thin layer: it starts the SDK join, emits the SDK events under the
existing event names, and copies the frames of aiortc tracks into the SDK.
This is a breaking change. This description lists the breaking changes compared
with 6.1.1.
MIGRATION_v6_to_v7.mdgives the details and the replacements.How it works now
flowchart LR subgraph app["Application"] code["application code<br/>handlers, add_tracks()"] srcs["aiortc tracks<br/>AudioStreamTrack, video tracks"] end subgraph py["getstream: pure Python"] join_fn["rtc.join()"] cm["ConnectionManager"] ps["ParticipantsState"] af["AudioForwarder"] vf["VideoForwarder"] end subgraph ext["getstream-rtc: PyO3 extension"] rcall["Call"] evs["EventStreams<br/>sfu, coordinator, client"] rtrack["RemoteTrack<br/>from TrackStream"] ltrack["LocalAudioTrack<br/>LocalVideoTrack VP9"] logq["bounded log queue<br/>and forwarding thread"] end subgraph sdk["Rust Video SDK on tokio"] joiner["coordinator join, SFU selection,<br/>retries, reconnects, migration"] pcs["SFU signaling,<br/>publisher and subscriber<br/>peer connections"] end coord[("Coordinator")] sfu[("SFU")] pylog["Python logging<br/>getstream logger"] code --> join_fn join_fn --> cm cm -->|"join, leave, update_subscriptions,<br/>publish, stop_publish"| rcall rcall --> joiner joiner <-->|"HTTP and WebSocket"| coord joiner --> pcs pcs <-->|"signaling and media"| sfu pcs --> evs evs --> cm cm --> ps cm -->|"participant_joined, track_added,<br/>audio, custom, call_ended,<br/>connection.state_changed, ..."| code pcs -->|"decoded remote media"| rtrack rtrack -->|"next_pcm(): audio event"| cm rtrack -->|"video_frames()"| code srcs --> af srcs --> vf af -->|"write_pcm()"| ltrack vf -->|"write_i420_at()"| ltrack ltrack -->|"paced and encoded"| pcs sdk -.->|"tracing records"| logq logq -.-> pylogConnectionManagermakes agetstream_rtc.Clientfrom the API key and thesecret (or the token) of the
AsyncStreamclient.asyncio. Its synchronous methods release the GIL.
ConnectionManageropens the event streams and the track stream before thejoin, so the events and the tracks of the join are kept.
the GIL. A log call never blocks. The first
connect()starts theforwarding thread; an import starts no thread.
Breaking changes
Compared with 6.1.1.
MIGRATION_v6_to_v7.mdgives the details and thereplacements.
Installation
webrtcextra installs the new packagegetstream-rtc(native extension). It has wheels only for Linux x86_64 and aarch64 (glibc 2.28 or later) and macOS arm64 (macOS 11 or later), and no sdist. On other platforms, for example Windows and Intel macOS,pip install "getstream[webrtc]"fails.webrtcextra no longer installsscipy,soundfile,websockets,websocket-client,structlog,tenacityandping3.rtc.join()andConnectionManagerrtc.join(call, user_id, create=True, subscription_config=None).**kwargsandmax_join_retriesare removed, and other keyword arguments raiseTypeError. Givedata,ring,notifyandvideotocall.get_or_create()beforertc.join().user_idhas no default.user_id=NoneraisesTypeError(wasValueError). A call without an id raisesTypeError.connection_statereturnsCallingState, not theConnectionStateenum, and is read only.CallingStateis not anEnum: its members have nonameorvalue, and you cannot iterate it.RINGINGandOFFLINEdo not exist.connect()raises (for exampleCoordinatorError) when the coordinator WebSocket cannot connect. 6.1.1 continued the join.connect(), noconnection.state_changedis emitted.connect()on the sameConnectionManageris not supported: no events and no tracks arrive. Use a newrtc.join()for each join.add_tracks()beforeconnect()raisesIllegalStateError. 6.1.1 logged an error and returned.wait()also returns when the call ends or when the SDK leaves the call: after a kick or a block, an SFUerrorwithDISCONNECT, an ICE failure, the rejoin rate limit, or too many failed negotiations.join_response,session_id,running,local_sfu,kwargs,ws_client,publisher_pc,subscriber_pc,publisher_negotiation_lock,subscriber_negotiation_lock,twirp_signaling_client,twirp_context,stats_reporter,tracer,reconnector,pc_id(),sfu_id(),republish_tracks(),drain_video_frames.start_recording(),stop_recording(),is_recording,get_recording_status(),recording_manager, the modulegetstream.video.rtc.recordingand the recording events. Use the server recording (call.start_recording()).Errors
getstream_rtcerrors, all subclasses ofRustError(ApiError,CoordinatorError,PermissionDeniedError,RtcError,ConfigError, ...), notSfuJoinErrororSfuConnectionError.rtc.join()andConnectionManager()raiseConfigError(orRustError) for an invalid client configuration, beforeconnect().add_tracks()raisesPermissionDeniedError,IllegalStateError,MediaErrororRtcError, notSfuRpcErrororTimeoutError.Events
getstream_rtcclasses (exported fromgetstream.video.rtc), not SFU protobuf messages:typeofTrackPublishedandTrackUnpublishedis nowtrack_type.track_typeandpublished_tracksuseTrackType. It is not anEnum; its members are equal to the protobuf integers and have the same hash.participantis aRemoteParticipant. It compares by identity (comparesession_id),joined_atis a Unix timefloat,customis adict, andtrack_lookup_prefixis removed.connection_qualityorsourcevalue that the SDK does not know becomes 0.TrackPublished.participant,TrackUnpublished.participantandCallGrantsUpdated.current_grantscan beNone.ice_restartarrives after the SDK has handled the SFU request.subscriber_offer,publisher_answer,ice_trickle,join_response,health_check_response,go_away,participant_migration_complete,connection_lost,reconnection_success,reconnection_failed. Watchconnection.state_changed. For the participant count, use the newparticipant_count_changed.participant_joinedis also emitted for the participants that are already in the call at join.call_endedis also emitted when the coordinator ends the call or the SDK leaves the call (reason0). It is emitted only once.track_publishedis not emitted for the tracks that existed before the join. Usetrack_added.errorevent, which is also the name of the SFUErrorevent.Remote media
track_addedgives aRemoteTrack, not(track_id, kind, user).add_track_subscriber()is removed: usetrack.next_pcm()andtrack.video_frames().video_frames()gives I420VideoFrames, not an aiortc track.ConnectionManagerreads every remote audio track for theaudioevent. Do not callnext_pcm()on these tracks.next_pcm(), for example byasyncio.wait_for()with a timeout, can lose the frame that it was about to return.Publishing
add_tracks(audio=...)andadd_tracks(video=...)replace the published track of the same kind. 6.1.1 added one more track. If two calls run at the same time, the second raisesMediaError.track_unpublished.s16frames are accepted.fltpframes stop the forwarding and unpublish the track.s16pframes are sent garbled.PcmRateMismatchErrorand unpublishes the track.AudioStreamTrackhaspace=Falseby default. In 6.1.1 it sent one frame per 20 ms. Code that sends it through its own aiortc connection needspace=True.audio_buffer_size_msbelow 20 raisesMediaError. A negative value raisesValueError.STREAM_PATCH_AIORTC_BITRATESandSTREAM_PATCH_AIORTC_H264_PRESEThave no effect.send-audioorsend-videois removed, the SDK unpublishes the track.Resampling
FrameResamplerandAudioStreamTrack.write()resample with the SDK windowed-sinc filter, not with PyAV:flush().flush()andwrite(..., final=True)add about 132 input frames of near-silence.s16.PyAVResampleris removed.participants_stateRemoteParticipantobjects keyed bysession_id, and it includes the agent itself.track_publisheddoes not change it and does not call themap()handlers.get_user_from_track_id(),get_stream_id_from_track_id()andset_track_stream_mapping()are removed.Subscriptions
role_filtersalso applies to the tracks that are published after the join.max_subscriptions, a new track can take the place of a track that is already subscribed.Removed modules and names
getstream.video.rtc:join_call_coordinator_request,JoinCallRequest,JoinCallResponse,ServerCredentials,Credentials,discover_location,HTTPHintLocationDiscovery,HEADER_CLOUDFRONT_POP,FALLBACK_LOCATION_NAME,STREAM_PROD_URL.getstream.video.rtc:peer_connection,pc,encoders_patches,signaling,twirp_client_wrapper,connection_utils(withSfuJoinError,SfuConnectionError,ConnectionState),models,reconnection,network_monitor,stats_reporter,stats_tracer,tracer,coordinator,location_discovery,recording.tracks.SubscriptionManagerandtracks.TrackType.track_util:patch_sdp_offer,fix_sdp_msid_semantic,fix_sdp_rtcp_fb,parse_track_stream_mapping,BufferedMediaTrack,VideoFrameTracker,detect_video_properties,AudioTrackHandler,PyAVResampler.Logging, telemetry and processes
getstreamlogger and its children, not fromaiortcandaioice. Records of other Rust crates are forwarded at WARNING and above.connect()reads the logger level.rtc.join()cannot use the SDK. Fork before the firstrtc.join(), or use thespawnorforkserverstart method.connect,leaveandrtc.add_tracksremain.Network
Installation and release
webrtcextra requiresgetstream-rtc>=0.1.0,<0.2.getstreamstays pure Python.getstream-rtchas its own version and its own release-please component (taggetstream-rtc-vX.Y.Z). Its first release is 0.1.0.getstream-rtcuses the crategetstream=0.2.0from crates.io. The crate links its own static libvpx and libopus, so a build from source needs only the Rust toolchain fromgetstream-rtc/rust-toolchain.tomland a C compiler.Tests and CI
getstream-rtc/tests/) and theConnectionManagertests (tests/rtc/) join real calls.cargo fmt --check,cargo clippy -D warnings,cargo test --lib. CI requires aCargo.lockthat matchesCargo.toml.rtc_wheels.ymlbuilds and checks thegetstream-rtcwheels on PRs that changegetstream-rtc/or the wheel build. It uploads nothing.release.ymlbuilds eachgetstream-rtcwheel from the tag, checks it and uploads it before it publishesgetstream.examples/rtc/echo_agent.pyandexamples/rtc/record_audio.pyuse only the publicgetstream.video.rtcAPI.Summary by CodeRabbit
BEGIN_COMMIT_OVERRIDE
feat(rtc)!: move the WebRTC transport from aiortc to the Rust Video SDK
BREAKING CHANGE: move the WebRTC transport from aiortc to the Rust Video SDK. See #303 and the migration guide for all changes. The main changes:
webrtcextra installs the new packagegetstream-rtc, which has wheels only for Linux x86_64 and aarch64 and macOS arm64rtc.join()takes onlycreateandsubscription_config;**kwargsandmax_join_retriesare removedconnection_statereturnsCallingStateand is read only;ConnectionStateis removedgetstream_rtcclasses, not SFU protobuf messages, andparticipantis aRemoteParticipantconnection.state_changedtrack_addedgives aRemoteTrackwithnext_pcm()andvideo_frames();add_track_subscriber()is removedgetstream_rtcerrors (subclasses ofRustError), notSfuJoinErrororSfuRpcErroradd_tracks()replaces the published track of the same kind, and video is sent as VP9AudioStreamTrackhaspace=Falseby default; setpace=Trueto send it through your own aiortc connectioncall.start_recording())Refs: #303
END_COMMIT_OVERRIDE