[swbiggart]

What this changes

At --tp 2, a Flash Next rank with no control-link frame yet waited in a blocking poll of up to 1 s and woke late
when the frame came; now a link wait polls with a 50 µs timeout for its first 50 ms, then blocks as before. Drafted
rounds are mostly such waits, so ranks now pick up the next header sooner. Replies are bit-identical; on two GB10s
drafted decode is 1.4-1.5% faster at one stream and 3.7-4.7% faster at four.

• Issue: n/a: a host-side wait, no new family, backend or arithmetic. Refs Roadmap: one N-rank layer for TP, PP and EP, shared by every family #2560 (multi-rank roadmap).
• Stacked on: main (ab78aef90). Commits: 6715e3fc7 (the change, its test, the serving pin) and a8c504622
  (changelog and ledger).
• atc overlap --json (ATC 0.9.6, 2026-11-11, on abb0ea372, before the rebase): complete: false, so I read all
  250 open pull requests' file lists. None touches cuda_rank_wire.zig; 100 share CHANGELOG.md,
  docs/optimization-ledger.md (server: --profile fast refuses as not planned (#1660) #2794, Flash Next on Metal: the scale index is the default; prompt gathers and grouped experts read it, routed arrays are released #2805, Windows: source-build the 5090 preview as tensorfold.exe #2809) or zig/tests/fixtures/gradient-serving.tsv (GLM-5.3-Flash on CUDA: a decode window's DSA select on many CTAs #2722, CUDA (GB10): a second carveout source, spark-attic's atticd, opt-in #2762, CUDA (GB10): Flash Next KV in the display carveout, opt-in #2763,
  cuda: tf_qmm_prefill_many works out its copy addresses once, before the K loop #2825, other rows). Trial merges against the old main and head: 26 clean, 52 already conflicted with main, 22 only
  at the CHANGELOG.md Unreleased heading the branch then re-opened (DeepSeek-V4.1: read the pack's weight spans past the page cache (core/direct_io) #1924, CUDA: #1764's BF16 prompt tile for Qwen ModelOpt, Nemotron and Dory, plus two exact Qwen prompt kernels (#1762) #2008, Vulkan: qmm_q4_tile_rows computes four rows and two columns of a prompt chunk per thread, 3.6% to 33% less run time on an M1 with the same logits (stacked on #2016) #2017, gemma4 Metal: Gemma 4 E4B (NVIDIA Nemotron 3.5 Bolt E4B) on M1 to M4, per-layer inputs and shared KV #2273, ricursive: Ripple-65B-Preview on native CUDA (NVFP4) #2302, ricursive: Ripple-65B-Preview's DFlash2 drafter on native CUDA #2475,
  ricursive: load Ripple's EXL3 packs through main's shared EXL3 code #2514, ricursive: serve Ripple-65B on two GB10s with up to 1M tokens of context #2522, ricursive: speed up Ripple-65B on two GB10s #2523, dsv41 lookup: add TF_DSV41_COPY_DRAFTER ranked band matching, doubling and yield gating #2638, hip flashnext: a shared round's streams draft in one head pass a step #2696, qwen27: slot 0 keeps its rewind point through the warm pass #2723, hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728, hip flashnext: an eight-row verify window, so a lone stream's copies land up to seven tokens a round #2730, XPU: Qwen3.8-27B attention merge with 16 subgroups per head, bit-identical #2799, server: the last --compact-at wins (auto then a fraction kept auto) #2883, server: serve --config FILE.toml reads flags from a file #2885, llama on Metal: Qwen2 checkpoints, Bimo-7B-CyberAgent's bf16 release with its q, k and v biases and YaRN at HF's rounding points #2890, xpu: a sampled Qwen3.8-27B row draws from a top-k kept on the card #2895, core: Admit and Ledger move to engine_admission.zig, unchanged #2896, server: --execution-witness serves a read-scoped snapshot of what runs, or why it is unknown #2901, native: the CUDA kept-state identity is written once through any hasher, its keys unchanged #2903): keep
  both lines. Re-run: atc overlap --json at 2026-11-12 14:07 UTC on head 608d85d30; every conflicts entry is named in this section, or adds no conflict beyond those it already has with main (1), or conflicts only in CHANGELOG.md's ## Unreleased lines, as this section describes (7); each checked with git merge-tree.

Checks on this head

• Head commit: a8c504622c5c1f7231ce501fbee0c4ddd569678a
• Receipts and GPU tests ran on the named builds before the rebases onto f0d8c8d50, eab305db0, 4f8317dac, 000b534a2, 3885dceb2, 0fed2e4a5 and ab78aef90,
  which moved only CHANGELOG.md lines and, onto ab78aef90, put this ledger row beside flashnext cuda: at --tp 2 a prompt pass reduce-scatters its routed experts' slots by rows and gathers their sums #2977's (main's Flash Next changes in that span, among them flashnext: refuse a malformed TF_FLASHNEXT_* setting by name before device work #2585 in eab305db0, which reads
  TF_FLASHNEXT_* settings, b0a3d76ba, which moves the n-gram gather's host workers, and flashnext cuda: at --tp 2 a prompt pass reduce-scatters its routed experts' slots by rows and gathers their sums #2977's expert exchange, touch none of these files), so every receipt stands.
• Lean checker (python3 tools/zig/lean_check.py), base ab78aef90 and head: 341 and 341, the same lines, 0 new.
  pr_check.py --base upstream/main: 0 errors, 1 advisory warning (no PR number).
• Tests run, with the command and pass, fail and skip counts (Apple M5 Max):
  • git diff --check upstream/main...HEAD and zig fmt --check build.zig zig: pass on this head; zig build: pass on a21bfddc2, this code before the changelog-only rebases onto 000b534a2, 3885dceb2 and 0fed2e4a5.
  • zig build test-cuda-host: pass on this head, Linux x86_64 in the CUDA 13.3.1 image (no error: lines; run without --summary, so no Build Summary line; with it
    on abb0ea372: 790/792 passed, 2 skipped); a planted expect(false) in the new test fails it.
  • zig build test test-golden -Dcpu=apple_m1, run on 85f61cf0f, this code on eab305db0 before the
    changelog-only rebase: 837/839 steps succeeded (1 failed); 7588/7662 tests passed (73 skipped, 1 failed). The
    failure, dashboard_test, is inherited: main eab305db0 fails the same test on the same Mac (837/839 steps succeeded (1 failed); 13041/13152 tests passed (110 skipped, 1 failed)).
  • zig build test-gradient-serving: pass on this head.
  • The new cuda_rank_wire test (on Linux it bounds context switches), three aarch64 runs each on a GB10: main's
    wait, no short polls, and short polls that never end each fail 3/3; this change passes 3/3. Windows skips it.
  • tf-cuda-test flashnext-rank on the RTX pair, abb0ea372: sum, eof, missing pass; timeout fails as on
    main (a test bug fixed separately).
• Rank CPU time, main 6c5cef894 against this change on it, two GB10s, three 4-stream drafted cells and 60 idle seconds: in the review reply of 2026-11-13. Per cell 0.5 s less on rank 0 and 0.1 s less on rank 1; idle unchanged.
• CI: runs on this head; it merges cleanly with main.
• Not run, and why:
  • Alternated RTX starts: the GB10 starts carry the claim.
  • GB10 8 streams and sampled: covered by the RTX starts.
  • 64k prompts: --context 65536 leaves no room to reply.
  • CPU cost of the short polls: at most 50 ms of 50 µs polls per wait, by design.
  • HIP, Windows, Metal: unreached (CUDA --tp 2 only).

Receipt

• Main against branch: 2 x GB10 (200G RoCE), alternated starts (fresh loads). Decode: median of 3 greedy
  256-token requests; streams: bench_concurrent.py --levels 1,4 --alone --serial, median of
  3 reps.

  Cell	main	branch	branch	main	change
  chat drafted	37.67	38.39	38.36	37.93	+1.5%
  chat plain	23.08	23.08	23.13	23.02	+0.2%
  code drafted	51.14	51.90	52.07	51.37	+1.4%
  code plain	23.16	23.10	23.16	23.07	+0.1%
  chat, 1 stream	37.8	38.3	38.4	37.6	+1.7%
  chat, 4 streams	100.2	106.0	105.5	101.8	+4.7%
  code, 1 stream	51.0	51.8	51.7	50.3	+2.2%
  code, 4 streams	143.0	150.9	149.3	146.5	+3.7%

  2 x RTX PRO 6000 Blackwell Max-Q, one start each, supporting only (chat / code):

  Cell	main	branch	change
  drafted	75.04 / 112.63	75.44 / 114.53	+0.5% / +1.7%
  plain	28.89 / 28.89	29.0 / 28.97	+0.4% / +0.3%
  8 streams, greedy	388.5 / 526.0	385.6 / 529.5
  8 streams, sampled	326.9 / 435.0	327.4 / 435.6

• Boxes it reaches: Flash Next on CUDA at --tp 2, both ranks.

• New switch? Default, accepted values, profile row: none.

• Environment, with the TensorFold commit, Zig and backend toolchain, machine and GPU, checkpoint and revision: GB10
  decode main 7d97cc656 against 043741fb9 (this change); GB10 prompt and RTX main 93a2e93ec against
  68758c594 and abb0ea372 (this branch). Zig 0.17.0, CUDA 13.3.1, NCCL 2.32.3. GB10: sm_121, 121.7 GiB, Ubuntu
  24.04.4, drivers 580.178.04 and 580.159.03. RTX: sm_120, driver 595.99.02. nvidia/Qwen3.8-Flash-Next-NVFP4
fc694b54fb0174e0913e6adf86691ef85a4ead47, MTP drafts, --context 65536 --prompt-cache-gib 0.

• Paths and platforms the change reaches, which I ran and which I could not: the CUDA --tp 2 rank link (ppoll on
  Linux, poll(1 ms) elsewhere); ran: both pairs above.

• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: every GB10
  decode request has the reference hash (chat 764da77fbd3f, code 4e759dbd859c) and every prefill size one hash
  (deae16afb5e0, e6b6ea31c756, a2dd25c04692); failed 0, alone 3/3 and 12/12, serial 1/1 in every cell.
  RTX: the same hashes on both, alone 24/24 at 8 streams, serial equal, greedy and sampled.

• Main against this branch, same machine and checkpoint, posted before asking for a run on our machines (or why you can't):
  above; raw JSON and tools: https://gist.github.com/swbiggart/520ab76ed681a684e2f774054c17d41e

• Decode speed before and after, from tools/bench_openai.py: GB10 cells: its stream client via a thin driver;
  bench_openai.py greedy on RTX: chat 70.0 / 69.5, code 74.6 / 76.6.

• Prompt speed before and after: cold, cached_tokens 0, alternated GB10 starts, prompt tok/s: 2,048 1,122.8 ->
  1,125.8 (+0.3%), 8,192 1,152.3 -> 1,153.6 (+0.1%), 32,768 1,148.0 -> 1,147.6 (-0.0%). bench_cache.py --require-cold qualified on both RTX starts.

• Licence of any new weights or drafter, or "n/a": n/a.

Checklist

• The title and description match the head commit, including the receipt and the stack.

• New tests fail before the change, pass after it, and skip cleanly without their dependency.

• Comments and docstrings are one line. No measurements or history in the source.

• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.

• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Resumed prompts not checked.)

• No precision traded for speed. If the bits change, the description says which sums change and why.

• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it, greedy and sampled (8 streams and sampled: one RTX start; 64k not run)

[ashhart]

Thanks @swbiggart, this is a small, well-aimed change, and the receipt is easy to follow.

What I checked:

• Only the host's wait changes. wait (zig/src/families/flashnext/cuda_rank_wire.zig:64-76) still returns at the same three points: data, its deadline, or the peer's error or hang-up. So no token can move, and your hashes agree with that.
• An idle server goes back to a blocking poll. Each wait starts its own 50 ms window (warm_ns, :62), and nothing sends frames while the ranks are idle, so a follower with no work blocks as it does on main.
• shortPoll (:79-86) treats an interrupted ppoll as an empty poll, so a signal can't end a wait early.
• The new test is checking the right thing. Main's wait, no short polls, and short polls that never stop each fail it, and this change passes.

Asks:

1. CI. CI hasn't run on any head yet, because CHANGELOG.md conflicts with main. Please merge or rebase onto today's main so it runs. We'll place the changelog line when we land.
2. CPU cost. The description says the short polls cost at most 50 ms of polling per wait, by design. Please post measured numbers: each rank process's CPU time over one 4-stream drafted cell, main against branch (from /proc/PID/stat or ps -o time). Please also post the same for 60 idle seconds after the last request.
3. Overlaps. flashnext cuda: at --tp 2 a prompt pass reduce-scatters its routed experts' slots by rows and gathers their sums #2977 lands ahead of this and adds its own row to the same table under item 25 in docs/optimization-ledger.md. flashnext cuda: at --tp 2 on one host, collectives run on a blocking split of the nonblocking communicator #2978 adds a row to that table too, plus a test at the end of cuda_rank_wire.zig, and re-pins that file in zig/tests/fixtures/gradient-serving.tsv. In each case, the PR that lands later keeps both sides (rows, link lines, tests) and re-pins from the merged file.

Your GB10 receipt stops at four streams, and 8 streams and sampled requests ran on one RTX start each. Once these are in and CI is green, we'll run main against this branch on two GB10s at 1, 4 and 8 streams, greedy and sampled, with token hashes.

• Synthena (Ash's Agent)

[swbiggart]

Thanks for the review.

1. Rebased onto today's main (0fed2e4a5) as b3f49c072, so CI can run. Only CHANGELOG.md conflicted. zig build test-cuda-host passes on this head, and lean is 341 against 341 on main, 0 new.

2. CPU time on two GB10s, one rank per host, main 6c5cef894 against this change on the same main. The patch matches the pushed head apart from the changelog line. Both rank processes were read from /proc/PID/stat (utime + stime). Each cell is tools/bench_concurrent.py --levels 4 --tokens 256 --reps 1 --temperatures 0: 4 drafted greedy streams, a code prompt and then a chat prompt. One start per build, three cells, then 60 idle seconds after the last request.

   	main rank 0	branch rank 0	main rank 1	branch rank 1
   CPU per cell, s (3 cells)	24.05, 23.23, 23.26	23.06, 22.92, 22.98	20.09, 20.13, 20.05	20.04, 19.95, 19.92
   Cores busy over the 3 cells	1.25	1.27	1.07	1.11
   CPU over 60 idle s, s	0.31	0.30	0.10	0.12

   The cells finish sooner on the branch (54.1 s against 56.5 s for all three), so each cell uses slightly less CPU in total. While a cell runs, a rank keeps 0.02-0.04 more of a core busy. Idle is unchanged: both ranks go back to blocking. Aggregate tok/s over the three cells: code 140.5, 143.3, 143.3 on main against 149.4, 148.5, 149.8 on the branch; chat 101.4, 100.4, 101.5 against 105.1, 106.0, 105.6.

3. Noted. Whichever PR lands later keeps both sides and re-pins cuda_rank_wire.zig from the merged file.

[swbiggart]

Rebased onto main ab78aef90, which has #2977, as a8c504622. The code commit is unchanged. docs/optimization-ledger.md keeps #2977's row under item 25 with this one beside it, and both link lines. Main hasn't changed cuda_rank_wire.zig, so its pin stands. On the new head, zig fmt --check, lean (341, 0 new), zig build test-gradient-serving and zig build test-cuda-host (Linux x86_64) pass.

[ashhart]

Thanks @swbiggart. The rebase keeps #2977's ledger entry, and the per-rank CPU measurements, including 60 idle seconds, close the remaining author asks.

This now waits on one final main/branch run that we'll schedule on two GB10s: 1, 4 and 8 streams, greedy and sampled, with token hashes and CPU time. No action needed from you. We'll preserve both sets of wire tests and re-pin the combined file when this and #2978 land.

• Synthena (Ash's Agent)