Skip to content

update simplew and simplew-tuned#700

Open
stratdev3 wants to merge 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew
Open

update simplew and simplew-tuned#700
stratdev3 wants to merge 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew

Conversation

@stratdev3
Copy link
Copy Markdown
Contributor

Description

  • simplew (production): default settings + AcceptPerCore ("Worker/thread counts matching available CPU cores")
  • simplew-tuned (tuned): recommended network settings for best performance.

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@stratdev3
Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew
/benchmark -f simplew-tuned

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 13, 2026

/benchmark -f simplew-tuned

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew-tuned | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,886,642 5328.0% 189MiB NEW NEW
baseline 4096 1,850,303 5405.7% 505MiB NEW NEW
pipelined 512 3,300,387 6048.8% 196MiB NEW NEW
pipelined 4096 3,337,334 6012.7% 356MiB NEW NEW
limited-conn 512 881,707 4429.8% 129MiB NEW NEW
limited-conn 4096 892,751 4036.6% 178MiB NEW NEW
json 4096 834,096 5773.7% 345MiB NEW NEW
json-comp 512 468,672 5783.2% 198MiB NEW NEW
json-comp 4096 477,210 5829.3% 394MiB NEW NEW
json-comp 16384 475,496 5627.0% 390MiB NEW NEW
json-tls 4096 570,251 6323.0% 1.1GiB NEW NEW
upload 32 1,792 2880.5% 2.3GiB NEW NEW
upload 256 1,917 6092.2% 6.3GiB NEW NEW
api-4 256 53,000 385.7% 159MiB NEW NEW
api-16 1024 140,577 1552.4% 300MiB NEW NEW
static 1024 71,040 4697.1% 907MiB NEW NEW
static 4096 73,656 4948.8% 1.4GiB NEW NEW
static 6800 79,619 5137.2% 1.9GiB NEW NEW
async-db 1024 177,906 3395.8% 377MiB NEW NEW
crud 4096 424,762 3857.5% 1.1GiB NEW NEW
echo-ws 512 1,941,701 5910.1% 217MiB NEW NEW
echo-ws 4096 1,784,108 5593.3% 560MiB NEW NEW
echo-ws 16384 1,522,305 5552.5% 2.5GiB NEW NEW
echo-ws-pipeline 512 2,951,921 6160.3% 190MiB NEW NEW
echo-ws-pipeline 4096 2,793,512 5827.8% 539MiB NEW NEW
echo-ws-pipeline 16384 2,702,416 5780.1% 783MiB NEW NEW
Full log
  WS frames:   13790258
  Latency samples: 13790200 / 13790257 responses (100.0%)

  WARNING: 18446744073709551615/13790257 responses (133766499592498.9%) had unexpected status (expected 2xx)
[info] CPU 5976.1% | Mem 249MiB

=== Best: 2951921 req/s (CPU: 6160.3%, Mem: 190MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    583us    361us    534us   1.35ms   8.33ms

  12501451 frames sent in 5.00s, 12494759 frames received
  Throughput: 2.50M req/s
  Bandwidth:  16.69MB/s
  WS upgrades: 596
  WS frames:   12494759
  Latency samples: 12494759 / 12494759 responses (100.0%)
[info] CPU 6086.8% | Mem 173MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.27ms    341us    409us   3.44ms   15.40ms

  13725678 frames sent in 5.00s, 13699107 frames received
  Throughput: 2.74M req/s
  Bandwidth:  18.34MB/s
  WS upgrades: 2008
  WS frames:   13699107
  Latency samples: 13699040 / 13699107 responses (100.0%)
  Latency overflow (>5s): 1
[info] CPU 6251.9% | Mem 362MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    858us    332us    393us   3.39ms   16.40ms

  13994309 frames sent in 5.00s, 13967564 frames received
  Throughput: 2.79M req/s
  Bandwidth:  18.69MB/s
  WS upgrades: 2188
  WS frames:   13967564
  Latency samples: 13967491 / 13967564 responses (100.0%)
  Latency overflow (>5s): 28
[info] CPU 5827.8% | Mem 539MiB

=== Best: 2793512 req/s (CPU: 5827.8%, Mem: 539MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.07ms    385us    553us   3.29ms   67.00ms

  11048905 frames sent in 5.00s, 11022076 frames received
  Throughput: 2.20M req/s
  Bandwidth:  14.76MB/s
  WS upgrades: 1677
  WS frames:   11022076
  Latency samples: 11022076 / 11022076 responses (100.0%)
[info] CPU 5652.4% | Mem 280MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.95ms    359us    430us   3.83ms    1.01s

  12907496 frames sent in 5.00s, 12855490 frames received
  Throughput: 2.57M req/s
  Bandwidth:  17.26MB/s
  WS upgrades: 3251
  WS frames:   12855490
  Latency samples: 12855490 / 12855490 responses (100.0%)
[info] CPU 6113.1% | Mem 537MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    388us    353us    406us   1.24ms   6.55ms

  13604442 frames sent in 5.00s, 13512081 frames received
  Throughput: 2.70M req/s
  Bandwidth:  18.22MB/s
  WS upgrades: 5773
  WS frames:   13512081
  Latency samples: 13512081 / 13512081 responses (100.0%)
[info] CPU 5780.1% | Mem 783MiB

=== Best: 2702416 req/s (CPU: 5780.1%, Mem: 783MiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 13, 2026

/benchmark -f simplew

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,978,811 5864.7% 202MiB +2.9% -0.5%
baseline 4096 1,904,591 5609.0% 669MiB +1.4% -0.3%
pipelined 512 5,950,477 5814.7% 255MiB +81.1% +17.0%
pipelined 4096 5,873,835 5865.1% 578MiB +80.4% +158.0%
limited-conn 512 883,751 4340.0% 127MiB +3.3% +2.4%
limited-conn 4096 869,736 3862.4% 277MiB -0.2% +3.7%
json 4096 821,973 5720.1% 271MiB -1.3% -19.1%
json-comp 512 470,727 5767.7% 195MiB +0.4% -3.5%
json-comp 4096 474,289 5646.9% 312MiB ~0% -2.2%
json-comp 16384 469,231 5759.6% 410MiB -0.7% -1.7%
json-tls 4096 325,687 3643.5% 1.1GiB -42.2% +10.4%
upload 32 1,793 2967.0% 2.3GiB +0.6% ~0%
upload 256 1,941 6213.7% 6.4GiB +0.9% -1.5%
api-4 256 54,392 386.0% 149MiB +2.0% -7.5%
api-16 1024 142,896 1552.9% 272MiB +1.6% -5.9%
static 1024 71,204 4782.8% 940MiB +7.1% +9.4%
static 4096 76,995 5071.4% 1.5GiB +3.3% +7.1%
static 6800 75,419 4946.8% 1.9GiB -2.0% ~0%
async-db 1024 177,891 3354.5% 389MiB ~0% +0.8%
crud 4096 425,600 3858.2% 1.2GiB +2.1% +9.1%
echo-ws 512 1,836,894 5581.8% 192MiB -4.7% +28.9%
echo-ws 4096 1,797,783 5855.9% 665MiB +0.8% -4.6%
echo-ws 16384 1,520,766 5823.7% 1.8GiB -2.1% -5.3%
echo-ws-pipeline 512 4,762,091 6028.4% 266MiB NEW NEW
echo-ws-pipeline 4096 3,922,915 5688.8% 835MiB NEW NEW
echo-ws-pipeline 16384 3,417,658 5219.9% 1.2GiB NEW NEW
Full log
  Throughput: 4.76M req/s
  Bandwidth:  31.78MB/s
  WS upgrades: 184
  WS frames:   23810458
  Latency samples: 23810363 / 23810458 responses (100.0%)
[info] CPU 6028.4% | Mem 266MiB

=== Best: 4762091 req/s (CPU: 6028.4%, Mem: 266MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew.json
httparena-bench-simplew
httparena-bench-simplew

==============================================
=== simplew / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    769us    249us    519us   3.89ms   85.00ms

  15301902 frames sent in 5.00s, 15246286 frames received
  Throughput: 3.05M req/s
  Bandwidth:  20.46MB/s
  WS upgrades: 3647
  WS frames:   15246286
  Latency samples: 15246286 / 15246286 responses (100.0%)
[info] CPU 5418.9% | Mem 335MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.23ms    215us    330us   2.54ms   31.30ms

  19368473 frames sent in 5.00s, 19309321 frames received
  Throughput: 3.86M req/s
  Bandwidth:  25.89MB/s
  WS upgrades: 4095
  WS frames:   19309321
  Latency samples: 19309285 / 19309321 responses (100.0%)
[info] CPU 5617.9% | Mem 604MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    902us    214us    315us   2.46ms   13.30ms

  19669875 frames sent in 5.00s, 19614579 frames received
  Throughput: 3.92M req/s
  Bandwidth:  26.29MB/s
  WS upgrades: 4096
  WS frames:   19614579
  Latency samples: 19614505 / 19614579 responses (100.0%)
  Latency overflow (>5s): 4
[info] CPU 5688.8% | Mem 835MiB

=== Best: 3922915 req/s (CPU: 5688.8%, Mem: 835MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew.json
httparena-bench-simplew
httparena-bench-simplew

==============================================
=== simplew / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.86ms    245us    446us   3.44ms   740.50ms

  15230472 frames sent in 5.00s, 15157368 frames received
  Throughput: 3.03M req/s
  Bandwidth:  20.37MB/s
  WS upgrades: 4596
  WS frames:   15157368
  Latency samples: 15157368 / 15157368 responses (100.0%)
[info] CPU 5106.7% | Mem 400MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.30ms    233us    401us   4.03ms   689.70ms

  16515024 frames sent in 5.00s, 16393760 frames received
  Throughput: 3.28M req/s
  Bandwidth:  22.12MB/s
  WS upgrades: 7835
  WS frames:   16393760
  Latency samples: 16393760 / 16393760 responses (100.0%)
[info] CPU 5432.6% | Mem 842MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.97ms    224us    378us   3.43ms   380.70ms

  17202006 frames sent in 5.00s, 17088294 frames received
  Throughput: 3.42M req/s
  Bandwidth:  23.03MB/s
  WS upgrades: 7107
  WS frames:   17088294
  Latency samples: 17088294 / 17088294 responses (100.0%)
[info] CPU 5219.9% | Mem 1.2GiB

=== Best: 3417658 req/s (CPU: 5219.9%, Mem: 1.2GiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew.json
httparena-bench-simplew
httparena-bench-simplew
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@stratdev3
Copy link
Copy Markdown
Contributor Author

the default settings are performing over the tuned ones ???

@stratdev3 stratdev3 marked this pull request as ready for review May 13, 2026 11:51
@stratdev3
Copy link
Copy Markdown
Contributor Author

Interesting, i'm looking forward to see the result in the website to properly compare the two flavors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants