[gilby]

What this changes

The Metal lane target (prompts, windows of several slots in one launch, keeps), the prompt cache with cached_tokens, and the native server over Metal (zig build dsv41-metal-serve); text only.

Part 16 of 18 of the stack for #1676; stacks on dsv41-metal-pr/15-serve-lanes (part 15, #1719), on main with parts 1 to 5 (#1705 to #1709) merged.

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: TensorFold 26f52c1 on main ad98b60 (native Zig 0.17.0; no MLX or PyTorch); Apple M3 Ultra (60-core GPU), 96 GB, macOS, for the kernel and device tests; Apple M3 Ultra (80-core GPU), 512 GB for full-model numbers; checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017.
• Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: the tiny model served through LaneHost in test-dsv41-metal-serve: concurrent == solo == serial, drafted == "draft": false, prompt-cache hits == fresh runs, seeded replies repeat, a cancelled prompt leaves the next request valid.
• Full model (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights; serve --parallel 4 --context 1048576 --prompt-cache-gib 8 --vision, DSpark drafts on; measured at the stack's top before the rebase onto main 3ce4a04, 60d51fd, the same 13 commits on main 9e8c368 (git range-diff shows each patch unchanged except part 10's one planted-control line, TF_CTL == 4 only, which the served kernel compiles out), whose served path for text is this part's): tools/bench_concurrent.py --tokens 128 --temperatures 0,1 --levels 1,4 --alone --serial: 8 cells, 0 failed, 0 unequal (alone 1/1 or 4/4 and serial 1/1 or 4/4 equal in every cell). The memory gate, 4 cold 912K-token prompts at once in 4 slots of 1,048,576 positions (all four recall their needle, about 333 GB of GPU memory at peak), is from the 29 Oct stack's top (5db73126) and was not re-run on this head.
• Real release weights (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release shards; run at the stack's top, a5e2ec9, whose Metal package and decoder layer are this part's): TF_DSV41_REAL=<checkpoint> zig build test-dsv41-metal-real on layer 2 (compress ratio 2) and layer 21 (ratio 1): exit 0, 5/5 steps, 4/4 tests each; every stage at 1 and 4 rows within its float64 bound, the closest margins at 1.000 of the bound (layer 2: shared SwiGLU 6.103e-5 vs 6.104e-5, routed + shared join 2.441e-4 vs 2.441e-4, hc post 3.906e-3 vs 3.907e-3; layer 21: shared SwiGLU 1.221e-4 vs 1.221e-4, join 4.883e-4 vs 4.883e-4, hc post 1.953e-3 vs 1.953e-3); mapped weights give the same output as copied weights, bit for bit, at 4 rows.
• Decode speed before and after, from tools/bench_openai.py: no Metal DeepSeek-V4.1 backend before this stack. After (60d51fd, the same patches before this rebase, --tokens 256 --reps 3 --temperatures 0,1, median): drafts on, fibonacci-raw 53.8 tok/s at T=0 and 50.5 at T=1, gpu-chat-no-think 43.8 at T=0 and 44.2 at T=1; drafts off (serve --no-drafts, one token a round), fibonacci-raw 35.4 at T=0 and 34.8 at T=1, gpu-chat-no-think 35.4 at T=0 and 34.6 at T=1. bench_concurrent aggregate at 1 / 4 streams: code 46.9 / 78.5 at T=0 and 52.1 / 61.4 at T=1; chat 43.2 / 76.1 at T=0 and 47.5 / 62.2 at T=1.
• Prompt speed before and after: no Metal DeepSeek-V4.1 backend before this stack. After (60d51fd, the same patches before this rebase): tools/bench_cache.py --prompt-file <32,768 token ids> --expected-prompt-tokens 32768 --require-cold --require-reuse --tokens 64 --repeats 2: cold first request ttft_s 42.76 (about 766 tok/s), cached_tokens 0; repeats 0.028 and 0.029 s and two concurrent 0.029 and 0.056 s with cached_tokens 32,768, the same tokens in every row; qualified.
• Tests run, with pass, fail and skip counts: zig build test tf-kernel-contract -Dcpu=apple_m1: exit 0, 627/627 steps succeeded; kernel contract, tf-kernel-contract --only dsv41 (Metal): SUMMARY 89 kernels: 89 pass gates 1-4, 0 fail, 0 known findings; 186 controls: 186 red as planted, 0 missed; 0 limit problems; gates 119142 ms, limits 161 ms; zig build test-dsv41-metal-all -Dcpu=apple_m1 (every DeepSeek Metal device step): exit 0, 9/9 steps succeeded; 37/37 tests passed; python3 tools/zig/lean_check.py: 5144 files, 424 problems, the same count as main (5027 files, 424). Zig's summary counts only the test binaries it ran in that invocation, so the step count is the comparable figure.
• Not run, and why: CUDA: no NVIDIA device here; Metal on M1, M2, M4 and M5: one M3 Ultra only; time-dsv41-metal-real (timing only) was not part of this receipt.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[gilby]

The full-model receipt in #1720 and #1722 is now from the stack's final head, 60d51fd7 on main 9e8c368b (Apple M3 Ultra, 80-core GPU, 512 GB; deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights; serve --parallel 4 --context 1048576 --prompt-cache-gib 8 --vision), replacing the numbers from the same stack before its rebase. The bodies carry the full lines; in short:

• tools/bench_concurrent.py --tokens 128 --temperatures 0,1 --levels 1,4 --alone --serial: 8 cells, 0 failed, 0 unequal.
• tools/bench_openai.py --tokens 256 --reps 3 --temperatures 0,1, medians: drafts on, fibonacci-raw 53.8 / 50.5 tok/s and gpu-chat-no-think 43.8 / 44.2 (T=0 / T=1); --no-drafts 35.4 / 34.8 and 35.4 / 34.6.
• tools/bench_cache.py --require-cold --require-reuse, 32,768 tokens: cold ttft_s 42.76 (about 766 tok/s), reuse 0.028 to 0.056 s with cached_tokens 32,768; qualified.
• Release weights at the same head: layers 2 and 21 against float64 4/4 each with mapped == copied bit for bit, and the vision tower 14/14.

Each number is within 1% of the pre-rebase run. The 4 x 912K memory gate is still the 29 Oct run, as the bodies say.

[ashhart]

Thanks @gilby. The code holds: no new kernel, the target refuses wrong slots, bad tokens, position and row mismatches, tree windows and non-prefix keeps, the lanes keep DeepSeek's opt-outs, the prompt cache is an exact token-prefix store under a byte cap, and CUDA is unchanged. Exactness is shown (concurrent = solo = serial, drafted = draft:false, cache hits = fresh; full model 0 unequal of 8 cells), and drafts are faster in all four single-stream cells (53.8/50.5/43.8/44.2 against 35.4/34.8/35.4/34.6 tok/s). One ask, a receipt, so never-slower can be checked: bench_concurrent at 1 and 4 streams with --no-drafts, and a 32,768-token cold first request with and without drafts. Small: adding logits to an existing entry can briefly exceed the cache cap, and the 4x912K memory gate isn't re-run on this head.

[ashhart]

Re-reviewed after the 05:25 push: the code is as we reviewed, rebased. The one open item is the never-slower receipt, which is listed with #1722's on #1722.

[ashhart]

Thanks @gilby. We compared this push with the version we reviewed. The only change is one build line, the lane test now importing core so it compiles on current main. The code stands as reviewed. The one open item is still the receipt, and we've listed it once on #1722 so it covers both. Two small points stay open, neither blocking: adding logits to an existing cache entry can briefly go over the cap, and the 4 × 912K memory gate is still the 29 Oct run.

[ashhart]

Thanks @gilby. #1720 and #1721 still apply cleanly to today's main, so neither needs a rebase; that part of our 5 Nov note is #1722's.

What's left before #1720 lands is the never-slower switch from that note. Your four-stream cells show drafts losing: 68.2 against 83.6 tok/s on code and 63.6 against 78.6 on chat at T=0. So:

• plain decode wherever drafts lose;
• 2- and 3-stream cells to place the switch;
• 1 and 4 streams rerun, with no drafted cell slower and token_sha256 equal.

#1720 is where serving starts, so please put the switch in #1720 itself, or tell us to land #1720 to #1722 together. If the switch uses the draft pricing that CUDA shares, scope it to Metal, or show a CUDA row that's unchanged.

Please also add a CHANGELOG line for #1720, citing "(#1720, part of #1676)". The two small points, the cache cap with logits added and the 912K memory gate, still don't block.

[gilby]

Updated at the head you asked for on 5 Nov. The switch is in, decided from the cells you wanted.

The switch (#1720, commit aaef961 + 52fb89885 + a832608 + bb41254b8): rounds with three or more active streams decode plain and skip the draft pass (TF_DSV41_DRAFT_PLAIN_STREAMS, default 3 on Metal, 0/off for CUDA — CUDA's rounds are unchanged, and the setting reads the same env var on both so a CUDA row can be shown). The 2- and 3-stream cells placed it: drafts win at 1 (47.0 vs 35.7 tok/s code T0) and 2 (58.2 vs 54.7), and lose from 3 streams (61.8 vs 71.9).

The never-slower question, honestly. The rule holds at 1 and 2 streams and does not hold yet at 3-4: with the switch on, plain-with-drafts-armed rounds run at 0.955-0.965 of plain (receipt8, fresh server per cell, --prompt-cache-gib 0, one M3 Ultra 512 GB, deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights; token_sha256 equal in every cell):

N	T	load	A (--no-drafts)	B (drafts, switch off)	C (drafts, switch default)	C/A
3	0	code	71.9	61.8	69.4	0.965
4	0	code	83.8	68.3	80.0	0.955
3	1	code	57.3	52.8	55.6	0.970
4	1	code	63.6	54.1	61.1	0.961

Two rounds of fixes narrowed it from 0.70 to ~0.96: the switch now skips the DSpark pass in plain rounds and does not launch one for the next round (52fb89885), and a stream's first (admission-time) draft ask waits for its first round (e361d1fbe). What remains is a per-round cost on the drafted path even when every window is plain — we are still narrowing it, and if it turns out inherent to keeping the drafted path armed, the honest reading is: the switch recovers 13 of the 15 lost points (C 80.0 vs B 68.1 at N=4 code T0), the remaining ~4% is the cost of having drafts armed, and we'd like your judgement on whether 0.98·A must include that.

Cells you asked for, receipts at the pushed head: 1 and 4 streams rerun with the switch in — drafted cells no slower than plain at 1-2 streams (C/A 1.22-1.49 at N=1), token_sha256 equal in every cell, 30/30 rep pairs equal. Full tables: receipt8 runs in the PR notes below (runs 2 and 3, summary.txt quoted).

Also in this push: CHANGELOG line citing "(#1720, part of #1676)"; the receipt script (receipt8.sh) and its summary format are in the repo's nodes/ tree, not this branch (say if you want them here).

[ashhart]

Thanks @gilby for the switch and for the honest table. The switch is well scoped: it counts streams instead of pricing, CUDA passes no default so its rounds are unchanged, and every new branch is gated on the setting. Some things need fixing before it lands. Several come from the rebase, which seems to have started from an older copy of the main commit than the one we reviewed on 4 Nov:

1. metal/forward.zig:248 and :252 call encodeEmbed and encodeHead without try. Both return errors, and metal/target.zig:360 is now the first caller of forward.run, so this PR on its own doesn't compile the Metal server or its serve tests. The fix is in dsv41 metal: image prompts #1722's first commit. It belongs here, as it was in the 4 Nov head.
2. Three more things from the 4 Nov head are missing. One is the Engram manifest check at boot (engram_manifest.check and its named errors, after rows_dir at metal/boot.zig:107). Another is the test-step wiring for serve_lanes_test.zig: no build step builds that file now. The third is the test-dsv41-metal-all step your receipt runs. Please restore them, or tell us why one should go.
3. The lean check fails. metal/boot.zig:98, :109, :118 and metal/entry.zig:19, :32 read settings with a direct getenv, which main has refused since 8 Nov. Please read them through platform/environment.zig; serve_lanes.zig:129 on main shows the form. Also, once merged with today's main, core/lanes/depth.zig is 601 lines and needs to lose one.
4. Please rebase onto today's main. Only CHANGELOG.md conflicts, but GitHub runs no CI while the PR conflicts, and none has run since 3 Nov.
5. On your question: yes, the bar includes the cost of keeping drafts armed. The default is what people run, so at every stream count it should be within 2% of --no-drafts. At 3 and 4 streams it is 3.0 to 4.5% behind. One place to look: plain rounds still run each slot's DSpark ingest (draft/lanes.zig:456), and each ingest is a separate command buffer the host waits on (metal/dspark.zig:153, :131), even though the drafter only reads its last window rows.
6. Please post the full table in the PR, at the pushed head: 1 to 4 streams, T 0 and 1, code and chat, plain and drafts-by-default, with token_sha256 for each pair. Please also name the commit it ran on. The commits your note cites (52fb89885, bb41254b8, e361d1fbe) aren't on the branch.
7. draft/lanes.zig:470-473 skips a stream's first draft ask whenever the switch is set, at one or two streams as well. But plain_streams_test.zig:127 says "below the threshold a stream drafts on its first ask", and its checks pass either way. Please either skip only when the round is at the threshold, or rename the test and include the 1- and 2-stream cells. Also, TF_DSV41_DRAFT_PLAIN_STREAMS accepts any size (draft/calib_env.zig:228) and is narrowed to 32 bits at draft/lanes.zig:124. Please bound it and refuse larger values by name.

The two small points (the cache cap when logits are added, and the 912K memory gate) stand as before and don't block. Main is frozen for today's release. With these in, #1720 to #1722 land in order after it.

• Synthena (Ash's Agent)

[3utsu]

Hi @gilby, @ashhart — first-time commenter here. The PR notes say Metal on M5 hasn't been run, so I ran this stack on an M5 Ultra (512 GB) and am posting the receipt in case it's useful. Happy to rerun anything in a different shape.

What this is and isn't. The current head de8495c3 doesn't compile the Metal server (the missing try at metal/forward.zig:248, as noted above), so everything below is from the 4 Nov reviewed commit 87436e18. That commit has no stream-count switch, so my "drafts" column corresponds to column B (drafts, switch off) in the 11 Nov table, not C.

Setup

• Mac Studio (Mac17,15), Apple M5 Ultra, 36-core CPU, 80-core GPU, 512 GB; macOS 27.2.1 (26B131); Xcode 27.2 (27B52); Zig 0.17.0; AC power, low-power mode off.
• Checkpoint: deepseek-ai/DeepSeek-V4.1-Flash-1017 at 08c8657f (weights byte-identical to 1bae0e10); all LFS files checked against the Hub sha256 (84/84). Engram row shards and token map built with tf-dsv41-engram-pack / tf-dsv41-token-map from this commit; verify passed.
• Server: TF_EXPERIMENTAL=deepseek tensorfold-dsv41 serve <dir> --host 127.0.0.1 --parallel 4 --context 1048576 --prompt-cache-gib 0 --no-update-check (plus --no-drafts for the plain cells). Text only: no --vision.
• Load: 34.4 s cold (4–6 s once the page cache is warm); log reports 284.2 GiB kept wired.

Tests on M5 (all at 87436e18)

• test-dsv41-metal 37/37, test-dsv41-metal-serve 7/7, test-dsv41-metal-all -Dcpu=apple_m1 37/37.
• tf-kernel-contract --only dsv41: 89 kernels pass gates 1-4, 0 fail, 0 known findings; 186 controls red as planted, 0 missed; 0 limit problems.
• Release weights, test-dsv41-metal-real: layers 2 and 21 against float64, 4/4 each, mapped == copied bit for bit.
• test-dsv41-metal-prompt-time: 2/2.
• Not run: time-dsv41-metal-real — it fails to compile at this commit (zig/tests/dsv41_metal/timing_real.zig:343 discards the error union from glue.mv; looks like a missing try). I did not patch anything.
• One build note: with Xcode 27.2, zig build tf-kernel-contract stopped with "missing Metal Toolchain" until I set TOOLCHAINS=com.apple.dt.toolchain.Metal.32023.922.4. No system setting changed.

Plain vs drafts, 1–4 streams — tools/bench_concurrent.py --tokens 128 --temperatures <T> --levels <N> --reps 3 --alone --serial, --prompt-cache-gib 0, three rounds; decode tok/s, median of the three rounds.

N	T	load	A (--no-drafts)	drafts (default, no switch)	drafts/A
1	0	code	45.1	58.9	1.312
1	0	chat	44.7	54.4	1.218
1	1	code	45.7	68.2	1.500
1	1	chat	45.7	62.4	1.362
2	0	code	70.1	79.5	1.133
2	0	chat	68.9	74.6	1.083
2	1	code	62.8	65.3	1.032
2	1	chat	62.7	68.1	1.090
3	0	code	93.2	85.6	0.918
3	0	chat	90.3	81.0	0.893
3	1	code	73.5	72.4	0.983
3	1	chat	72.5	71.8	0.993
4	0	code	110.8	93.6	0.845
4	0	chat	106.0	89.3	0.841
4	1	code	83.9	77.3	0.899
4	1	chat	85.0	75.8	0.893

• token_sha256 equal in every pair (plain vs drafts, concurrent vs alone vs serial); 0 failed, 0 unmeasured, across all 48 server runs.
• Same shape as on the M3 Ultra: drafts win at 1 and 2 streams and lose from 3. So a threshold of 3 looks right on M5 as well.
• Round-to-round spread is about 3% (the third round ran ~3% faster for both modes; I did not log temperature). The ratio moved by ±0.01 between rounds, except N=4/T=1/code (0.893–0.930).

Cold 32,768-token first request — tools/bench_cache.py --require-cold --tokens 64 --repeats 1 --streams 1, fresh server each time, three rounds:

• drafts: ttft_s 37.46 / 39.47 / 38.62
• --no-drafts: ttft_s 36.61 / 37.82 / 38.90
• All qualified, prompt_tokens 32,768, cached_tokens 0, the same token_sha256 in all six runs.

4 × 912K memory gate, re-run — same server line with --prompt-cache-gib 8, still without --vision:

• Four prompts of 933,888 tokens each (912 × 1,024), each with one needle at a different depth, sent at the same time. All four needles recalled.
• Peak GPU allocation 315.6 GiB (338.9 GB); /health memory.peak 329,715,351,552 B. No swap-outs during the gate, memory pressure normal throughout, minimum free 34.4 GiB.
• The four prefills ran one after another, about 1,190 s each (≈783 tok/s), 4,788 s in total.

Where my run differs from the receipt you asked for

• A fresh server per (round, N, T, mode); code and chat shared a server within that cell. I can redo it with a server per load.
• No --vision, so the memory peak may read lower than a run with the vision tower loaded.
• I read "912K" as 933,888 tokens.
• This is the pre-switch commit. Once the head compiles again I'll rerun the same matrix there, so the C column (drafts with the switch) can be read on M5 too.

Raw JSON for every cell, the server logs and the memory trace are kept; I can attach them or put them in a gist if that helps.

[ashhart]

Thank you @3utsu, and welcome. This is a careful receipt, and an M5 Ultra with 512 GB is a machine we can't run DeepSeek-V4.1 on ourselves. A few things it settles for the 4 Nov commit (87436e18) on M5:

• The Metal kernel contract (89 kernels, 186 of 186 controls red) and the release-weight layer checks (layers 2 and 21 against float64, mapped equal to copied) hold on M5, not only on the M3 the PR was built on.
• The 4 × 912K memory gate passes with all four needles recalled at a 315.6 GiB peak and no swap-outs. It's good to have that number from a real 512 GB machine.
• At one and two streams, drafts beat --no-drafts everywhere you measured (1.03-1.50x).

Your offer to rerun on the fixed head is the most useful next step. Once @gilby pushes a head that compiles (the try in metal/forward.zig:248 from our note above), the same matrix there would let us read the C column (drafts with the stream-count switch) at 3 and 4 streams on M5. One change would help: a fresh server per load (code and chat separately), so prefix reuse within a server can't mix the two. Please put the raw JSON and the server logs in a gist and link it.

@gilby, 3utsu also found a second missing try: zig/tests/dsv41_metal/timing_real.zig:343 discards the error union from glue.mv, so time-dsv41-metal-real doesn't compile. Please fix it with the forward.zig one. The Xcode 27.2 toolchain note (TOOLCHAINS=com.apple.dt.toolchain.Metal.32023.922.4 for tf-kernel-contract) might be worth a line in the DeepSeek Metal recipe.

• Synthena (Ash's Agent)

[3utsu]

Thanks for the welcome! As offered, here is the raw data behind the 87436e18 receipt above: https://gist.github.com/3utsu/3aaeded2e964558f0cc34ce2de3fc861

It has the full bench_concurrent.py output for all 48 servers, the six cold 32K runs, the 4 × 912K gate (results, /health samples and a 15 s memory trace), every server log, the test logs and the scripts that drove the runs. Only home paths and the host name are shortened.

Once the head compiles, I'll rerun the same matrix there with a fresh server per load (code and chat separately) and add it to the same gist.

[ashhart]

Thanks @3utsu, that's everything we asked for: the per-server JSON, the logs and the scripts that drove them. We'll read the C column on M5 from your rerun once the head compiles.

• Synthena (Ash's Agent)

[3utsu]

The rerun I offered, at #1720's head c97d661c4 (#1722's 3ce0d75d6 is this plus its own 5 commits). Same Mac Studio M5 Ultra 512 GB as before (macOS 27.2.1, Xcode 27.2, Metal toolchain 32023.922.4), deepseek-ai/DeepSeek-V4.1-Flash-1017 release weights, serve --parallel 4 --context 1048576 --prompt-cache-gib 0, tools/bench_concurrent.py --levels N --tokens 128 --reps 3 --temperatures 0,1 --alone --serial. A is --no-drafts, C is the default; one server per arm at 1 and 2 streams, A, C, C, A at 3 and 4, as in @gilby's M5 table. Two differences from that table: a fresh server per load (code and chat on separate servers, 24 servers), and no --vision (this head answers an image part with 400 "text parts only"; images arrive with #1722).

C/A, this run against @gilby's M5 Ultra at 3ce0d75d6:

N	T	load	A	C	C/A	gilby C/A
1	0	chat	46.7	56.9	1.218	1.209
1	0	code	46.8	61.3	1.310	1.301
1	1	chat	46.2	61.3	1.327	1.353
1	1	code	46.3	69.5	1.501	1.500
2	0	chat	71.1	77.1	1.084	1.083
2	0	code	72.5	82.2	1.134	1.129
2	1	chat	64.5	71.0	1.101	1.088
2	1	code	65.4	67.5	1.032	1.027
3	0	chat	93.4, 93.3	93.1, 93.2	0.998	0.997
3	0	code	96.5, 96.4	96.3, 96.4	0.999	0.997
3	1	chat	75.1, 74.9	74.8, 74.6	0.996	0.999
3	1	code	76.3, 76.3	76.3, 76.3	1.000	0.998
4	0	chat	109.5, 106.5	109.3, 109.2	1.012	0.998
4	0	code	114.4, 114.4	114.0, 114.0	0.997	0.998
4	1	chat	85.5, 82.9	85.0, 84.9	1.009	0.999
4	1	code	86.6, 86.9	86.5, 86.7	0.998	0.999

• token_sha256 is one hash per stream across every A and C server in every cell, concurrent equals alone equals serial, 0 failed. The hashes are also the same as in my 87436e18 run.
• Every cell at 3 and 4 streams is at 0.996 or better. The 4-stream chat cells read above 1 because the last A server was about 3% low while another desktop app was busy on the machine; it is in that server's snapshots.
• Thermal pressure read moderate (ProcessInfo "fair") in 58 of 89 cell snapshots and nominal in the rest, with no pmset warning. @gilby's cells were all nominal, so please read the absolute tok/s with that in mind.

Cold 32,768-token first request, tools/bench_cache.py --require-cold, fresh server each, A, C, C, A: 38.07, 40.04, 40.56 and 38.91 s, cached_tokens 0, all qualified. C's first token is about 1.8 s (4.7%) behind A's here. In my 87436e18 run the two overlapped, and these four ran while thermal pressure was moderate, so I can't separate the cause. This is not @gilby's prompt: none of the three ways I tried to build one gave 0e3f2ebade35, so I used the prompt from my 87436e18 run (its reply hash 24c37bc44666 is the same at both commits). If you share the 32K ids, I'll rerun with them.

tools/bench_openai.py on the default server (256 tokens, 3 reps, median): code 70.5 tok/s greedy and 65.2 sampled, chat 56.2 and 55.9.

Tests at this head: test-dsv41-metal-all 9/9 steps, 38/38; test-dsv41-metal-real layers 2 and 21, 4/4 each; time-dsv41-metal-real builds and passes 2/2; kernel contract 89 kernels, 186 of 186 controls red.

Raw JSON, server logs, per-cell thermal and load snapshots and the scripts are in the same gist, as files named *c97d661c*, with a new README section: https://gist.github.com/3utsu/3aaeded2e964558f0cc34ce2de3fc861

[ashhart]

Thanks @gilby, and thanks @3utsu for the independent rerun at c97d661c4 itself.

The copy-window fix is right and small. plainCopies (zig/src/core/lanes/windows.zig:236-242) only acts when cfg.plain_streams is set. Only DeepSeek's lanes set it (draft/lanes.zig:122), and only the Metal server passes a default (serve_lanes.zig:124), so other families and CUDA are unchanged. Grammar masks are attached after allocate (core/lanes/engine.zig:264-265), so a constrained stream whose copy becomes a plain window keeps its masks. The two tests in drafted_copy_test.zig pin both sides of the threshold.

The receipts agree. On the M3 Ultra (the fix on a43ad7b98, windows.zig byte-identical, so no rerun at c97d661c4 is needed), on your M5 Ultra at #1722's 3ce0d75d6, and on @3utsu's M5 Ultra at c97d661c4, every cell at three and four streams is at 0.996 or better against --no-drafts. Drafts lead at one stream (1.21x to 1.50x) and at two (up to 1.13x), and token_sha256 is equal across arms in every cell. The 4 x 912K gate passes on M5 at #1722's head, which contains this one. On the cold 32K prompt, your M5 run is level between arms (34.2 to 34.9 s). @3utsu's 4.7% gap ran under moderate thermal pressure with a different prompt, so we read the prompt path as level. If you can share the 32K ids with @3utsu, a matched run would be nice to have, but it doesn't hold this up.

On the plain-streams switch: keep it in this PR as reviewed. It doesn't change tokens, it only applies to DeepSeek on Metal, and it's what keeps the default from ever being slower than --no-drafts today. You're right that main's guide treats a count switch as a bug rather than a limit, so we'll treat it as a known one with a fix on the way: when the priced rule you're tracking in #2984 passes the same gate (within 2% of --no-drafts in every cell, ahead where confidence is high), the pull request that brings it turns TF_DSV41_DRAFT_PLAIN_STREAMS's default off and removes the switch. No need to hold the stack for it.

Against today's main, only CHANGELOG.md conflicts, and we'll resolve that while landing. This is ready. It lands first, then #1721 and #1722 in order.

• Synthena (Ash's Agent)

[ashhart]

Thank you @gilby. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @gilby, and @3utsu for the independent M5 Ultra receipt! This is in TensorFold 1.2.8, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.8

• Synthena (Ash's Agent)