Repository navigation
[Doc] Refresh the benchmark numbers - #144
Merged
Merged
Conversation
Re-measured on an Apple M4 Pro with pygrpc-native 0.8.0 and grpcio 1.85.1, and added payload size, server streaming, token streaming and 30-second results to the benchmark README. Signed-off-by: MinJun Choi <mj04300017@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Updates the Apple M4 Pro numbers in the README and benchmarks/README.md to pygrpc-native 0.8.0 and grpcio 1.85.1. The "about 1.5x" and "about 5x" claims still hold (1.45x and 5.08x).
benchmarks/README.md also gets results for payload size, server streaming, the token streaming command and a 30-second run. At 1 MiB, grpc sync is faster than every pygrpc-native server, and the doc says so.
The 40-vCPU Linux sentence and the Modal table weren't re-measured. The README sentence now says it was pygrpc-native 0.1.0.
Fixes #143
How it was tested:
Apple M4 Pro (14 cores), macOS 27.0.1, Python 3.12.15, with pygrpc-native 0.8.0 and grpcio 1.85.1 installed from PyPI into a fresh venv. benchmarks/echo.py from main, five runs per case, rotating the order of server kinds each round, on AC power with nothing else running. Tables show medians. pmset -g therm showed no thermal warnings before or after each batch.
Special notes for your reviewers:
The sync servers top out near 3,200 msgs/s in the token streaming case because echo.py gives them a 64-thread pool. That's noted under the table.
If applicable: