[ashhart]

What this changes

Since 0b44bd1, Qwen3.8-27B with DFlash2 on M5 Macs sends one greedy stream through the 16-row lane loop, where copies are off. A copy-heavy reply, such as a whole-file rewrite, lost its long-copy tail: about 7 tokens a round instead of about 47, and about 70% slower. Tokens were exact.

After this change, an eligible lone greedy request at the M5 16-row default uses the long-copy driver again. Every copied token is still verified by the target. Any arrival hands the stream back to the lane loop. Sampled requests, shared rounds, wider windows and M1-M4 Macs keep the lane loop, as on main.

Three changes keep prose and code level with main on this path:

• The driver proposes the lane path's tree, 16 siblings at the root instead of 4.
• It adds copy rows only after a 24-token suffix match (the lane proposer's confident match) instead of 5 tokens.
• The dependent picks, accepted taps and head copy are encoded in the verify's existing tail, as the lane path already does, instead of in a separate command buffer.

Explicit settings keep their meaning. TF_QWEN27_LANES=1 keeps every request on lanes. TF_QWEN27_LANES=0 selects the lone driver with its old proposal (4 siblings, 5-token match, separate submission) on either device class. No kernel, arithmetic, budget, EOS or sampling change.

• Issue: Fixes Qwen3.8-27B + DFlash2 on Metal: one greedy stream's copy-heavy output is ~70% slower since the 16-row lane loop became the default (tokens exact) #3013
• Stacked on: main (6c5cef8). Two commits: 8dd73ff restores the driver, 394ef08 adds the proposal, copy and verify-tail changes.
• atc overlap --json: incomplete (time limit). Merges cleanly with lfm2: DSpark drafting with owned context and confidence scheduling #1394, Implement GLP control vectors with a fused Metal kernel for Qwen3.8-27B #1835, XPU 27B decode attention: tree rows of a key slice run back to back (bit-identical, +23 % at 32K, +33 % at 131K) #2595, fix(qwen27): warn when Metal startup exceeds available memory #2886, dsv41 metal: prompts in pieces between the other streams' rounds (stacked on #1720) #2950, flashnext cuda: at --tp 2 a prompt pass reduce-scatters its routed experts' slots by rows and gathers their sums #2977, flashnext cuda: at --tp 2 on one host, collectives run on a blocking split of the nonblocking communicator #2978 and flashnext cuda: the opt-in --tp 2 decode graph takes the dense split and keeps its graph across drafted verifies #2982. Text conflicts with qwen27: serve the 27B pair, refuse what a pair cannot keep, and die with the peer #1434 (1 file) and Qwen3.5-9B DFlash drafter in Zig engine #1851 (3 files). Seven others not checked.

Checks on this head

• Head commit: 394ef08 (tree e3497c87377c62f6e887b43adc156b03ced15ebb)
• Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: 345 on the base, 345 on this head, 0 new.
• Tests run, with the command and pass, fail and skip counts:
  • zig fmt --check on the 9 changed Zig files: clean.
  • zig build test-qwen27-window-host test-qwen27-facts-host test-qwen27-draft-tree-host test-qwen27-bank-boundary-host test-qwen27-equality-host test-qwen27-pass-host test-qwen27-keep-host test-qwen27-startup-host test-qwen27-reply-cache-host test-qwen27-shared-allocation-host test-qwen27-grants-host check-qwen27-speed-host: 45/45 steps succeeded; 505 passed, 0 failed, 1 skipped. Window 20, facts 18, draft tree 57, bank boundary 4, equality 2, pass 9, keep 3, startup 2 + 2, reply cache 3, shared allocation 360 + 1 skipped, grants 25; the speed check compiles the served host.
  • Red runs when the branch was prepared: the new window expectation fails on main's policy and passes here, and the new selector test catches a 4-sibling mutant.
  • On the M5 Ultra Mac, tf-qwen27-dflash-served-check and tf-qwen27-streams pass on both main and this branch. The streams check compares 252 shared BF16 logit rows with solo over 25 rounds.
• CI: not run yet.
• Not run, and why: Metal and GPU tests on the host (the device evidence is the receipt below). M1-M4 and M5 Macs other than the Ultra: no such machine was used. Full no-GPU CI.

Receipt

• Main against branch: main 6c5cef8 against this tree (e3497c8737), on the same machine and checkpoint. One stream at T 0 and T 0.7 (top-k 20, top-p 0.95, seed 1234), and four simultaneous greedy streams, on three cases: whole-file rename (a 5,421-token rewrite), plain prose and new code.

Streams	Case	T	Main tok/s	Branch tok/s	Change	Range main / branch
1	whole-file rename	0	240.99	791.87	+228.6%	240.84-241.21 / 791.54-794.53
1	whole-file rename	0.7	238.45	238.15	-0.1%	238.07-238.95 / 237.65-238.36
1	plain prose	0	129.29	129.09	-0.2%	129.00-129.68 / 128.84-129.43
1	plain prose	0.7	120.50	121.41	+0.8%	120.40-120.53 / 121.34-121.69
1	new code	0	339.89	339.21	-0.2%	338.82-340.83 / 338.72-339.43
1	new code	0.7	325.07	324.80	-0.1%	324.77-325.69 / 324.62-324.99
4	whole-file rename	0	198.20	211.90	+6.9%	197.19-199.18 / 208.76-212.78
4	plain prose	0	65.17	66.78	+2.5%	65.06-65.98 / 66.54-66.95
4	new code	0	124.49	124.80	+0.3%	123.50-125.94 / 123.73-125.29

No cell is slower with separated ranges. The three small one-stream losses overlap main's range. On the rewrite, tokens a round go from 6.84 to 46.73. Main with TF_QWEN27_LANES=0 gives 795.72 tok/s on the rewrite (794.42-796.18), and 0b44bd1's parent gave 796.1 in #3013.

• Boxes it reaches: Metal Macs with GPU tensor units (M5 and later) at the default 16-row window.
• New switch? None. TF_QWEN27_LANES keeps its values (unset, 0, 1). Only its unset meaning on M5 at 16 rows changes.
• Environment, with the TensorFold commit, Zig and backend toolchain, machine and GPU, checkpoint and revision: main 6c5cef8 and this tree e3497c8737. Zig 0.17.0, ReleaseFast engine. An M5 Ultra Mac with 256 GiB, Metal, one rank. TensorFold/Qwen3.8-27B-MLX-4bit (affine 4-bit, group 64) at revision 70ae7fac63, DFlash2 drafter at revision 50307d4c4c. Context 16384, thinking off, prompt cache off, slot resume off.
• Paths and platforms the change reaches, which I ran and which I could not:
  • Ran: the served lone greedy driver on an M5 Ultra; sampled and four-stream requests, which stay on lanes.
  • Same as main by construction: shared lane rounds (selection returns the same options), M1-M4 defaults, wider windows, and TF_QWEN27_LANES=0 or 1.
  • Not run: M1-M4, other M5 chips, TF_QWEN27_LANES=0 on this head.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: every reply has the same token sequence for its case and temperature: main and branch, one and four streams, measured and warm-up. That is 996,834 tokens: 664,556 measured and 332,278 warm-up, including the workaround runs. So four streams equal one. This receipt compares with main's drafted replies, not with "draft": false.
• Main against this branch, same machine and checkpoint, posted before asking for a run on our machines (or why you can't): the table above.
• Decode speed before and after, from tools/bench_openai.py: the table above. It comes from a served-request driver, not tools/bench_openai.py. Each side ran two fresh server processes, in main, branch, branch, main order. Each case discards one complete warm-up reply, then keeps two complete natural-stop replies a stream. Rates are per-request native decode rates, as medians of four waves. Four-stream rates are per request, not aggregate.
• Prompt speed before and after: n/a. The change picks the decode loop after the prompt pass. Client time to first token was recorded: on the rewrite, 2.961 s on main and 2.960 s here. On the short prompts the ranges overlap.
• Licence of any new weights or drafter, or "n/a": n/a.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. The lone stream runs in the lane core's own lone mode (core/lane_host_lone.zig), as it did before 0b44bd1 changed the M5 default: the lane host installs the driver for one lone greedy stream and takes the stream back on any arrival. Sampled, constrained and shared work stay on the lane rounds; TF_QWEN27_LANES=0 keeps its meaning.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Concurrent equals solo, and every reply equals main. Not compared with "draft": false; slot resume was off.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it, greedy and sampled. Measured at one and four streams with the drafter, greedy and sampled. The widest admitted width and runs without the drafter were not measured.