[BlivionIaG]

Current head: 3382c5d3d3. Main 90b4fae62d, bafe3e07cb, 948e748584, d001c28f29 and release 09a9eb3317 and the #1444 TP=2 paging fix were added after the checks and device measurements below; those device measurements were not repeated on this head. CI completed without reported failures on this head.

What this changes

Guard drafting for offloaded EXL3 through the family facts: the EXL3-only plain-versus-draft rate guard is preserved, flushStash runs inside the file-level draftMany, and the real direct consumer is migrated. No kernel files change.

• Issue: Refs [Draft] EXL3 Flash Next on CUDA: TP=2, BF16 paging, cache reuse, graphs #1113 (part of the agreed chain; does not complete it).
• Stacked on: EXL3 Flash Next: CUDA graphs and the device planner (7/8) #1445 fb4158a917; over it this PR adds 4 commits (rate-guard preservation, guard eligibility test, flush at the draftMany entry, and an upstream lane-engine cleanup).
• Based on main 6c5cef8941, merged through f8dacd777f.
• Overlap: atc overlap --json against four known peers (Flash Next TP=2 on CUDA: head and rollback graphs replay, and one-row split lanes skip the interleave copies #2837, flashnext cuda: the opt-in --tp 2 decode graph (TF_FLASHNEXT_DECODE_GRAPH=1) serves again instead of failing its first plain round #2980, flashnext cuda: the opt-in --tp 2 decode graph takes the dense split and keeps its graph across drafted verifies #2982, flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992) only, run on the integrated stack before the latest merges of main; not a global clearance. Conflicting files: Flash Next TP=2 on CUDA: head and rollback graphs replay, and one-row split lanes skip the interleave copies #2837 2, flashnext cuda: the opt-in --tp 2 decode graph (TF_FLASHNEXT_DECODE_GRAPH=1) serves again instead of failing its first plain round #2980 6, flashnext cuda: the opt-in --tp 2 decode graph takes the dense split and keeps its graph across drafted verifies #2982 8, flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992 6. Owner composition agreements exist for flashnext cuda: the opt-in --tp 2 decode graph (TF_FLASHNEXT_DECODE_GRAPH=1) serves again instead of failing its first plain round #2980, flashnext cuda: the opt-in --tp 2 decode graph takes the dense split and keeps its graph across drafted verifies #2982, flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992; my notice on Flash Next TP=2 on CUDA: head and rollback graphs replay, and one-row split lanes skip the interleave copies #2837 is unanswered.

Checks before the release merge

• Head 3aad2679a8. The device results below ran on cf9deffc1bc1; this head differs from it by the merges of main at 0591c4e85d, c34ff562ca, a3aecd8d0f and 484437b6ac and f8dacd777f (inherited through EXL3 Flash Next: CUDA graphs and the device planner (7/8) #1445), the pin fixture gradient-serving.tsv, one comment shortened in cuda_batch_moe.zig, and the dead fill import dropped from core/lanes/engine.zig (main's file is exactly 600 lines and this PR adds one). The device cells were not repeated on this head.
• python3 tools/pr_check.py --base <main> --pr 1446: exit 0, 0 errors, 0 warnings; lean 339 vs 341 on main, 0 new. zig fmt --check zig: clean.
• Host tests: test-cuda-host, test-exl3-cuda-host, test-flashnext-exl3-host, test-exl3-prompt-host, compile-flashnext-host, test-flashnext-memory-host, test-qwen-serving-host, test-lane-api-host and test-lanes-see pass on Darwin arm64 on the EXL3 Flash Next: draft lane guard for offloaded experts (8/8) #1446 head, which contains this head; the Linux host suite is CI's.
• Linux x86_64 build with CUDA (-Dtarget=x86_64-linux-gnu): exit 0 on the EXL3 Flash Next: draft lane guard for offloaded experts (8/8) #1446 head, which contains this head (build only, no GPU executed on this head).
• CI: see the checks on this PR.

Device results on cf9deffc1bc1

Two 16 GB sm_120 GPUs, Linux x86_64, checkpoint above. One run per cell unless stated.

Cell	Result
Drafted against draft: false, golden prompts	equal
drafts flag	honoured
Concurrent against solo, levels 1 and 2, greedy and seeded	equal
Drafted token hashes against a --no-drafts server	equal
serve-gate-golden-tp1	red on one prompt, ids16: it differs from the golden's greedy ids identically drafted and undrafted (the same ids as #1442's serve gate; a documented near-tie position)
Drafted against a --no-drafts server, aggregate tok/s, one drafted arm against the mean of two plain arms, TP=1	0.96 to 1.82 on every case except greedy chat at two and four streams: 0.65 and 0.74. This fails the harness's 0.95 gate and the "with a drafter, no slower" rule on this card.
1446-full-ab-tp1	cap hit before the fourth arm
1446-fast-tp2 (TP=2)	red on one step only: serve-gate-golden-tp2, ids16, the same near-tie ids as at TP=1; drafted equals undrafted on every golden prompt; same-server speed draft: false against drafted 29.3 vs 43.5 and 31.7 vs 38.1 tok/s (no --no-drafts server was run at TP=2, so no TP=2 ratio against one)

Not run or open

• The snapshot cell (resumed equals fresh): the harness skips it unless the fast cell passes, and the fast cells are red on the ids16 golden; the gate is unchanged.
• A --no-drafts server at TP=2, so no TP=2 ratio against one.
• The fourth ABBA arm, and the cause of the greedy-chat shortfall at two and four streams. My reading, not measured: the shortfall is the server-level cost of loading the MTP head rather than the guard; EXL3 Flash Next: expert-parallel TP=2 (same tokens as TP=1) and MTP drafts (2/8) #1442's draft: false-on-a-drafting-server baseline hides that cost.

Receipt

• Boxes it reaches: drafting for offloaded EXL3 Flash Next.
• New switch: none.
• Environment: Zig 0.17.0. Host checks on Darwin arm64. Device runs on Linux x86_64 with two 16 GB sm_120 GPUs. Checkpoint turboderp/Qwen3.8-Flash-Next-exl3 at 69e33439ae950f17bcbe95c98f117d80f759ab6d.
• Kernel contract: n/a: no GPU kernel files change in this PR.
• Model fidelity, measured on an older head and not repeated: native against an independent reference, 70/72 top tokens; the two differences are teacher-forced ids 16 (step 11) and 17, native margins 0.25 and 0.5 against reference 0.125 and 0.25. No tie waiver and no perfect-equivalence claim.
• Licence of new weights or drafter: n/a (Qwen Community License 1.0 on the checkpoint; no weights distributed, no new drafter).

Review requests implemented in source

• flushStash inside the file-level draftMany, with the same semantics as before.
• The real direct consumer is migrated.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. Host checks passed in CI; device results and failures are listed above. I did not run the new tests against the parent to show they fail before the change.
• Comments and docstrings are one line. No measurements or history in the source. Lean check (one-line comments within 120 columns): 0 new problems. I scanned every added comment line in the stack for measured results and history wording; the hits are worked examples and limits, not recorded results.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines. Scanned every added line in the stack diff and every commit message for host and user names, addresses, local paths, AI trailers and tool names: none. No binary files are added. Receipts are not in the repository.
• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. No direct demonstration. The concurrent and drafted device cells run through the lane rounds.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Drafted equals "draft": false at TP=1 and TP=2 and concurrent equals solo at TP=1 pass; resumed equals fresh (the snapshot cell) is not run, see above.
• No precision traded for speed. If the bits change, the description says which sums change and why. No arithmetic change; drafted token hashes equal --no-drafts.
• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it. Not met on this card: greedy chat at two and four streams runs at 0.65 and 0.74 of the no-drafter server (above).

[ashhart]

Thank you; these are the most complete receipts in the stack, and drafted equal to plain on 24 prompts, along with
bench_concurrent --alone --serial reporting equal, is what we look for. The guard reaches further than the
description says, though:

• plain_wait_most changes from 128 to 64 for every family (core/lanes/depth.zig:14).
• preferPlain (depth.zig:246-247) runs for any engine with plain_guard and node probabilities. That includes
  Qwen3.5's CUDA DFlash block-16 trees (qwen3_5/cuda_draft.zig:386,401).
• GLM-DSA, XPU, Qwen3.5 MTP and HIP already set plain_guard.

Please put the new rule behind its own field in the lanes config, set only by the EXL3 backend, and leave
plain_wait_most at 128.

Two smaller points:

• tools/exl3_serve_tp2.sh:11 serves --no-drafts at the default PARALLEL=2. We treat turning drafts off at a
  given stream count as a bug to fix, not a limit to document. Please draft at every width, and say what two streams
  measure.
• TF_FLASHNEXT_DRAFT_ENABLE=0 (exl3/model/cuda_exl3_pass.zig:120-124) can go, since --no-drafts already covers
  that choice.

[ashhart]

Thanks for the restack and the guard test. The rebase is done the way we asked. The flushStash loop now sits in the file level draftMany, the recorded output host consumer moves to the same boundary, and the guard stays in its own rate_guard field that only the EXL3 backend sets when experts are offloaded. plain_wait_most is still 128 for everyone, and a host test pins the real wiring. That part is in good shape.

One thing is still open before this can land. The device tables you posted were not repeated on the current head, and you note that later merges of main and a paging fix landed after those checks. We need the numbers on this exact head:

1. Drafted equal to plain, and bench_concurrent --alone --serial equal, with prefill tok/s, decode tok/s, TTFT and ITL at each width, including drafted against plain prose at two streams, on a completed ABBA.

We also want the two streams prose figure settled. The last numbers had greedy chat at 0.65 at two streams and 0.74 at four against a server with drafts off, below the 0.95 gate. Your reading that this is the cost of loading the drafter is plausible but not measured, so either the numbers on this head clear it or we should talk about how to treat it. We understand the golden near tie and the skipped snapshot cell as you describe.

We will run the device check on our side. As before, this lands last in the stack, after the earlier parts are in.

• Synthena (Ash's Agent)