[SergiioB]

What this changes

attn_decode on the Qwen3.8-27B XPU path now runs on a (rows, heads, slices) grid instead of (slices, heads, rows), so the tree rows of one key slice run back to back and share its K/V in L2. Per-work-group arithmetic is unchanged: outputs and tokens are bit-identical to main. The pair attn_decode + attn_merge is registered in the XPU kernel contract as a replacement that must equal today's kernels byte for byte.

• Issue: Refs XPU Qwen3.8-27B: decode attention re-reads the KV cache once per tree row (11 tok/s at 131K); proposed fix +48 % #2543 (the gate for this first part is agreed there; the wider slices come later as an opt-in).
• Stacked on: main (branch base 1a5644892; merges cleanly onto fcf85f7).
• atc overlap --json: complete: false (limitations otherPullRequestsLimit, pullFilesUnavailable, timeLimit), no overlaps reported. Manual check of the 200 most recent open pull requests: only shared build/doc files overlap (zig/build/xpu.zig, zig/build/kernel_contract.zig, docs/kernel-contract.md in lfm2: DSpark drafting with owned context and confidence scheduling #1394, EXL3 Flash Next: host-resident K/V on the tier (6/8) #1522, xpu: LFM3-VL-2B on the Intel GPU (Iris Xe, no DPAS): pictures in, text out #1838, Metal: the 27B verifies 32 rows a pass on pre-M5 Macs #1841, MoE route tap: optional TF_ROUTE_TAP trace of global top-k #1886, RoPE: Qwen3.8-27B on XPU from core's exact helper (#1852 batch 8; 1-ulp value changes) #2000, Living Weights on Qwen4 dense, step 2: the native CUDA and Metal learner (experimental, --slide refused) #2054, Living Weights on Gemma 4, step 2: the native GeGLU learner on CUDA and Metal (experimental, --slide refused) #2062, kimi: experimental K3.1 index operators on Metal, native serving stays capped #2228, qwen3-asr: the XPU binding on Gen12LP (#2105, stacked on #2239) #2255). Each of those already conflicts with current main on its own; none touches decode.cl, forward.zig or the contract sources added here.

Checks on this head

• Head commit: a1d896a88 (6 commits: grid order, contract registration, NAVIGATION line, widths 2/4/8/12, contract doc, receipts).
• Lean checker on base and head: 116 problems on 1a5644892 (5,936 files) and 116 on this head (5,939 files); 0 new findings.
• Tests run:
  • tf-xpu-contract-run --device 0 (BMG-G31), rerun on this head's contract sources with the exit status recorded: SUMMARY bmg-g31 0xe223 10 kernels: 10 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems, exit 0. Attention: 20 shapes bit-equal to the kernel it replaces.
  • xpu-kernel-contract-limits: every registered launch fits bmg-g31, every record current, every control red.
  • Host: zig build test --summary all on Linux x86_64 at this head: 607/607 steps succeeded, 7,925 of 7,991 tests passed, 66 skipped, 0 failed (exit 0). Attention registration tests 19/19; test-kernel-contract 222 executed + 13 cached, 0 failed. test-golden and fuzz suites were not run locally.
  • zig build --summary all (plain build) at this head: 33/33 steps succeeded, exit 0.
  • xpu-qwen27-tests on one B70, main and this branch: seam pass (0/248,320 differing logits at cuts 64 and 1,024) and close pass on both. means fails on both, identically: it prints group means: worst error / Σ|terms|: prompt 5.12e-5, decode 1.86e-5 (bound 2e-6) and then exits 139 (segfault), on unmodified main and on this branch. The group-means/GEMM path is unchanged here; this PR does not fix it. The "all device tests pass" part of the gate is open pending your call on this pre-existing failure.
• CI: no CI results yet at the time of writing; see the checks on this pull request.
• Not run: XPU_KV_INT8=1; --parallel above 1 (multi-stream rounds); resumed prompts from the prompt cache (--prompt-cache-gib 0 throughout); sampled decoding; any GPU other than BMG-G31; QPARTS != 1 builds.

Receipt

• Main against branch: 8 cells, code and prose prompts at 68 / 7,976 / 32,552 / 130,856 prompt tokens (prose: 52 / 7,963 / 32,540 / 130,846), 512 tokens out, greedy, thinking off, one stream, --parallel 1 --context 139264 --prompt-cache-gib 0, one discarded warmup, drafted and "draft": false.
• Boxes it reaches: Intel XPU, Qwen3.8-27B decode and verify rounds (≤ 16 rows). Prompt rows (> 16) use attn_prompt, unchanged.
• New switch: none.
• Environment: this branch and main 1a5644892, both built with Zig 0.17.0 and ocloc 26.05.37020 plus one local build change that drops -cl-intel-128-GRF-per-thread from the Gemma 4 prompt-rows kernels (ocloc 26.05 rejects it; not part of this PR). Ubuntu 26.04, kernel 7.0, xe driver, oneAPI 2026.1. Intel Arc Pro B70 (BMG-G31, 32 GB), 230 W cap. Checkpoint TensorFold/Qwen3.8-27B-MLX-4bit @ 22d8d538, drafter z-lab/Qwen3.8-27B-DFlash2, bf16 KV. The whole-model branch binary was built at the tree of 42473ce29 (grid order and NAVIGATION line); later commits change only contract sources, docs and receipts, with production decode/forward unchanged.
• Exactness (token_sha from the server log of each request):

Prompt tokens	main drafted	main "draft": false	branch drafted	branch "draft": false
52	596a68a0fce5 ×2	×1	×2	×1
68	a6b86fdf8992 ×2	×1	×2	×1
7,963	3cb4af2530db ×2	×1	×2	×1
7,976	46ebf41df2f3 ×2	×1	×2	×1
32,540	150f6d329941 ×2	×1	×2	×1
32,552	4bf356e770cb ×2	×1	×2	×1
130,846	8d68ee4e9421 ×1	×1	×2	×1
130,856	4fd3ed8293b5 ×1	×1	×2	×1

All four columns carry the same token_sha in every row (×n = requests). The retained server logs cover 46 of the 54 timed depth requests: main drafted has token_sha for n=2 below 130K tokens and n=1 at 130K tokens, while its speed medians use n=3 and n=2; those 8 earlier main requests have output-text hashes only, which also match. Short prompts through tools/bench_openai.py: 9ccc5ea1a3d7 (fibonacci-raw) and 703e2fba4510 (gpu-chat-no-think) on both, 12 requests each side.

• Decode speed, streamed tokens/s after the first token, code / prose, drafted, rounds alternated main/branch (main n=2–3, branch n=2 per cell, median):

Prompt tokens (code)	main	branch	change
68	128.2 / 102.5	128.1 / 102.4	0 %
7,976	85.1 / 67.8	90.1 / 71.8	+6 %
32,552	37.3 / 27.6	45.9 / 33.9	+23 %
130,856	11.4 / 10.2	15.1 / 13.6	+33 %

"draft": false (n=1 each): 32.0 / 30.4 / 23.4 / 12.8 (main) and 32.0 / 30.4 / 23.7 / 13.0 (branch). At 130,856 tokens main's drafted decode (11.4) is slower than its own "draft": false (12.8); on this branch drafted (15.1) is faster than "draft": false (13.0) again. tools/bench_openai.py --tokens 256 --reps 3 --temperatures 0, two alternated rounds: fibonacci-raw 131.9 / 131.9 (main) and 131.8 / 131.8 (branch); gpu-chat-no-think 87.8 / 87.8 and 87.7 / 87.8.

• Prompt speed: n/a. Prompt rows use attn_prompt, which this change does not touch; cold TTFT at 130,856 tokens is 133.9 s on both.
• Licence of new weights or drafter: n/a.

Kernel contract

• Registered: attn_decode followed by attn_merge as one operator (bf16 Q rows, bf16 K/V cache, tree table → bf16 attention rows), replacing the 1a5644892 pair, whose source is kept in zig/kernels/xpu/contract_attn_upstream.cl and run on the same inputs. Gate 3 is byte equality at 20 shapes: tree widths 1, 2, 4, 8, 12, 16 (heap-order parents) and visible keys 33, 34, 8,193, 32,768, 32,769, 32,770. Equality is shown for the combined operator's bf16 output, not for the fp32 partials separately; chains and other tree topologies are not fixtures.
• Planted control: a merge that drops the last non-empty slice; it fails as planted (bytes differ from the upstream pair from byte 0 at 1×6144×33).
• SUMMARY bmg-g31 0xe223 10 kernels: 10 pass gates 1-4, 0 fail, 0 known findings; 6 controls: 6 red as planted, 0 missed; 0 limit problems, exit 0.
• Chips not run: everything except BMG-G31.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. The contract's planted control fails as planted and the replacement passes; I did not record the new registration against a broken dispatch or without a device.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Drafted equals "draft": false at every cell above; resumed-prompt and concurrent equality were not run.
• No precision traded for speed. The bits do not change.
• Decode and prompt processing are no slower than main at one stream and the widest admitted; with --drafter, no slower than without it, greedy and sampled. One stream greedy only; the widest admitted and sampled runs were not done.

Revert: git revert the six commits; no format, switch or public interface changes.

[ashhart]

@SergiioB, the grid-only change preserves each row's arithmetic, slice order, subgroup reduction and BF16 store. The replacement contract and the matching main/drafted/plain hashes on the B70 support that argument. This keeps the wider-slice proposal separate, as agreed in #2543.

The agreed device gate is still open: means fails and exits 139 on both versions. Since it fails the same way on main, and this change's outputs are byte-identical, we'll track that failure as its own issue rather than hold this change on it; the bound stays as it is. Please open that issue with the isolated main and branch commands, the full failure logs (both the means bound and the exit 139) and exact build identities, and link it here.

Please also make the performance receipt reproducible: retain the public prompt inputs, comparison command or script, raw per-request results, full build/checkpoint/drafter revisions and binary hashes, including the compiler workaround used on both sides. The current summaries omit token digests for eight timed main requests. An optimization-ledger note should identify this as the grid-only change and retain the stated unrun cases. Multi-stream and int8-cache qualification remain deferred as agreed, rather than passed.

[Mirrdhyn]

Independent run of this PR on a second Arc Pro B70 (PCIe 5.0 x16), including the three paths your receipt lists as not run, in case it helps the review.

Setup. Your 6 commits cherry-picked onto main 1bb0ea270 (no conflicts; nothing under zig/src/xpu or zig/kernels/xpu changes between 1a5644892 and 1bb0ea270), against 1bb0ea270 itself. z-lab/Qwen3.8-27B-DFlash2 drafter, greedy, thinking off, 256 tokens out, --prompt-cache-gib 0, 1 warmup then 3 timed requests per cell (min-max within ±0.1 tok/s unless noted). Drafted and "draft": false on the same server.

Exactness. Every cell below gives the same reply sha256 on main and on the branch, and drafted equals "draft": false in every cell:

• GPTQ 4/128, bf16 KV, code and prose at 4 depths (16 cells);
• the same with XPU_KV_INT8=1 (16 cells);
• TensorFold/Qwen3.8-27B-MLX-4bit, your checkpoint, bf16 KV (16 cells);
• --parallel 4 --context 36864: one request alone, then the same request among 3 concurrent ones (overlap checked), at ~8K and ~29K, on both arms.

Decode tok/s, drafted, main → branch (code / prose):

prompt tokens	GPTQ, bf16 KV	MLX 4-bit, bf16 KV	GPTQ, XPU_KV_INT8=1
34 / 29	141.6 → 141.3 / 80.9 → 80.8	140.3 → 140.1 / 81.7 → 81.6	138.3 → 138.3 / 84.0 → 84.1
7,588 / 7,912	59.1 → 62.3 / 61.2 → 65.3	56.3 → 59.3 / 74.8 → 79.6	64.1 → 61.5 / 75.0 → 72.2
28,946 / 30,966	29.5 → 36.4 / 31.2 → 38.9	31.4 → 38.7 / 33.6 → 41.6	33.8 → 35.8 / 35.7 → 38.2
121,022 / 119,739	10.6 → 14.5 / 9.2 → 12.5	10.4 → 14.2 / 11.0 → 14.9	11.5 → 13.2 / 10.5 → 12.0

"draft": false is unchanged within noise in every column (+0 to +1 %). The MLX column matches your receipt's shape (+23–24 % at ~30K, +36 % at ~120K). Two things you may want to look at:

• with XPU_KV_INT8=1 the gain is smaller (+6–7 % at ~30K, +14 % at ~120K), and at ~8K the branch is 4 % slower on both prompts (main 64.0–64.1 vs branch 61.5–61.6 over the 3 timed runs, code; 75.0 vs 72.2, prose), reproducible across runs;
• at ~120K, drafted stays slower than "draft": false on main in all three columns (e.g. 10.4 vs 13.5, MLX code) and roughly level with it on the branch (14.2 vs 13.7).

Reading the diff. Only the get_group_id mapping in attn_decode and the launch order in forward.zig (x = T, y = 4, z = ns) change; each work group computes and writes the same PM/PL/PO entries for the same (t, head, slice), so bit-equality holds by construction, which the hashes confirm on the int8 and multi-stream paths too. The QPARTS != 1 branch gets the same mapping.

Not run here: sampled decoding, QPARTS != 1 builds.

[ashhart]

@SergiioB, thanks. The diff is a pure grid remap, and each work group still writes the same partials, so we accept the bit-exactness argument. Thanks also to @Mirrdhyn for the independent B70 run: matching hashes on GPTQ, MLX and XPU_KV_INT8=1, drafted equal to "draft": false, and concurrent equal to solo at --parallel 4. That covers the int8 and multi-stream exactness cells we'd deferred.

Before landing, please:

1. Open the means issue (commands, full logs including exit 139, build identities) and link it here.
2. Complete the receipt: prompt inputs, the comparison script, raw per-request results, full revisions and binary hashes on both sides including the ocloc workaround, and digests for the 8 main requests that lack them.
3. Add an optimization-ledger entry naming this as the grid-only change, with its unrun cases.
4. Add a CHANGELOG line ending "Thanks to @SergiioB."
5. Add a planted control that launches the new attn_decode with the old {ns, 4, T} grid and shows it red, so the contract also guards the launch order forward.zig depends on.
6. @Mirrdhyn measured the branch about 4% slower than main at ~8K with XPU_KV_INT8=1 (64.1 → 61.5 and 75.0 → 72.2 tok/s). Could you reproduce that on your card (main against branch, n ≥ 3) and either address it or explain it with numbers?

[Mirrdhyn]

Numbers for item 6, and a data point for item 1, from the same B70 (PCIe 5.0 x16). Stack since my first run: kernel 7.0.0-43, compute-runtime 26.35.39758.21, IGC 2.41; same binaries (main 1bb0ea270, branch = your 6 commits on it, 68a07473e). GPTQ 4/128 + DFlash2, --parallel 1 --context 132096 --prompt-cache-gib 0, 256 tokens greedy, 1 warmup + 3 timed requests (spread ≤ ±0.1 tok/s).

Item 6: with XPU_KV_INT8=1 the branch is slower below ~12K tokens and faster above, drafted only. Code prompt, drafted, main → branch:

prompt tokens	main	branch	change
3,822	78.2	76.3	−2.3 %
7,588	64.1	61.5	−4.1 %
15,022	54.5	55.9	+2.5 %
28,946	33.8	35.7	+5.8 %
58,543	22.6	25.2	+11.9 %

"draft": false is unchanged at every depth (e.g. 31.72 → 31.75 at 8K, 20.35 → 20.54 at ~59K); same sha256 on both arms and drafted equal to "draft": false in every cell; at 8K the prose prompt shows the same −3.8 %. With bf16 KV the branch is ahead at every depth I measured (8K +5–7 %, 32K +23 %).

Per-kernel GPU time (dev runner XPU_PROFILE, --bench=7588, 8-row chain rounds, synthetic tokens; compares arms, not served absolutes), drafted rounds at ~8K:

ms per round	int8 main	int8 branch	bf16 main	bf16 branch
attn_decode	5.87	6.70 (+14 %)	7.86	6.32 (−20 %)
attn_merge	1.91	2.25	1.86	2.17
other kernels	unchanged		unchanged

So at short context with 256-byte int8 rows the new order costs more than it saves in attn_decode, and the saving grows with the number of key slices. I tried keeping the old order for an int8 cache (cap > 0): it restores main at 8K (64.12) but gives back the int8 gain at 32K (33.81), so a rule on the cache type alone is wrong; one on int8 and few slices would fit these numbers, and I'd measure it before proposing it.

Item 1, means: on 1bb0ea270 with the stack above, the means test of xpu-qwen27-tests passes here, three runs out of three, exit 0: group means: worst error / Σ|terms|: prompt 4.19e-7, decode 2.80e-7 (bound 2e-6). The exit 139 in your receipt may depend on the environment (your ocloc 26.05 workaround and oneAPI 2026.1 differ from mine); worth stating in the issue.

What I'm looking at next, in case it helps you or overlaps with your own plan: the bits are identical and the diff only remaps group ids, so this points at memory locality rather than at the kernel's code. Two hints: attn_merge, unchanged, slows by the same ~0.3 ms a round in both modes, which suggests the order the partials are written in matters; and on main the int8 attn_decode is only ~25 % faster per round than bf16 while reading half the bytes, so the int8 path may not be bandwidth-bound at these depths. I'll check the two kernels' ISA (ocloc disasm), L2 behaviour and achieved bandwidth (VTune) on int8 at ~8K in both orders, and report back here.

[ashhart]

Thanks @SergiioB, and thanks again @Mirrdhyn for the follow-up: the depth sweep and the per-kernel times make the XPU_KV_INT8=1 picture much clearer, and the means pass on your stack is a useful data point for that issue.

Nothing has changed on the branch since our last comment, so the list from then still stands:

1. Add a CHANGELOG line ending "Thanks to @SergiioB."
2. Add an optimization-ledger entry naming this as the grid-only change, with its unrun cases.
3. Open the means issue with the commands, the full logs (the bound and the exit 139) and the build identities, and link it here. @Mirrdhyn's run passes means on main with a newer compute runtime and no ocloc workaround, so please include your toolchain versions and the workaround, and say whether it still fails without them.
4. Add a planted control that launches the new attn_decode with the old {ns, 4, T} grid and shows it red. Today zig/tests/xpu/contract_attn.zig:100 launches each kernel with its own grid and the only control is the dropped slice (zig/kernels/xpu/contract_gate_controls.cl:94), so nothing guards the launch order zig/src/xpu/forward.zig:344 relies on.
5. Complete the receipt in docs/receipts/xpu-qwen27-attn-grid-order/: the prompt inputs, the comparison script, raw per-request results (not only medians), full revisions and binary hashes for both sides including the ocloc workaround, and digests for the 8 main requests that have none.
6. The XPU_KV_INT8=1 slowdown. @Mirrdhyn now shows the branch 2–4% slower below about 12K tokens with an int8 cache (drafted), with attn_decode 14% slower a round at ~8K, and faster above that. Please reproduce it on your card (main against branch, n ≥ 3) and then either fix it or show with numbers why it should stand. Choosing the grid per launch, for example keeping the old order for an int8 cache with few slices, would keep every work group's arithmetic and so the bits. If you go that way, the contract needs to cover both orders, with measurements at each side of the switch point.

We can't run XPU ourselves, so your B70 numbers and @Mirrdhyn's are the device evidence. Once these are in, we'll approve the CI workflows on your new head. When CI is green, it can land after the current release.

[ashhart]

Thanks @SergiioB, and thanks again @Mirrdhyn for the B70 runs. We've done the items on our list that don't need an Intel card, as two commits on top of yours on this branch:

• Planted control (item 4). A new planted control launches the new attn_decode on the old {ns, 4, T} grid (zig/tests/xpu/contract_attn.zig:14, :108-109; registered at zig/tests/xpu/contract_run.zig:61). It uses the old grid only where that grid stays inside the buffers, meaning no more slices than rows. With more slices than rows, the old grid would read query rows and the tree past their ends. Gate 3 should go red at 16 rows over 33 keys (2 slices), the second served shape. docs/kernel-contract.md:116 and :127-129 describe it.
• Optimization ledger (item 2). A note under item 24 (docs/optimization-ledger.md:419-424) names this as the grid-only change. It gives the B70 numbers, including the XPU_KV_INT8=1 short-context cost, and lists the cases not run: sampled decoding, resumed prompts, QPARTS != 1 builds and any GPU but BMG-G31.
• CHANGELOG (item 1). We'll add the line when this lands.

We have no Intel GPU, so we can't run the control. We built the contract runner against stand-in images to check that it compiles. Please run tf-xpu-contract-run on your B70 and confirm the summary reads 7 controls: 7 red as planted.

These still need your card:

1. The XPU_KV_INT8=1 slowdown (item 6). This is what keeps the PR from landing, since we don't land regressions. @Mirrdhyn measured the branch 2-4% slower than main below about 12K tokens with an int8 cache (drafted), with attn_decode 14% slower per round at about 8K. Please reproduce it (main against branch, n ≥ 3) and then fix it or show with numbers why it should stand. Choosing the grid per launch keeps each work group's arithmetic, and so the bits. One way is to keep the old order for an int8 cache with few slices. If you do that, the contract has to cover both orders, with measurements on each side of the switch point.
2. The receipt (item 5). It needs the prompt inputs, the comparison script, raw per-request results, full revisions and binary hashes for both sides (including the ocloc workaround), and digests for the 8 main requests that have none.
3. The means issue (item 3). Please open it with the commands, the full logs (the bound and the exit 139), the build identities, your toolchain versions and the ocloc workaround, and link it here. @Mirrdhyn's run passes means on main with a newer compute runtime and no workaround, so please also say whether it still fails for you without the workaround.

Once these are in, we'll approve CI on the new head.

• Synthena (Ash's Agent)

[Mirrdhyn]

The follow-up I promised on 9 Nov on the XPU_KV_INT8=1 slowdown (item 6). It is about the mechanism only: I have not measured a fix, and it is not an argument for landing the regression. Same B70, stack and binaries as my last comment (main 1bb0ea270, branch 68a07473e), same dev-runner setup as the per-kernel table there (~7.6K tokens, 8-row chain rounds).

ISA. ocloc disasm of attn_decode gives the same instruction stream on both sides (1,732 instructions, 256 GRF, SIMD16, no spills) except six group-id reads. So between these two binaries the int8 gap comes from the dispatch order, not from code generation or registers.

Memory (VTune gpu-hotspots, global memory accesses, two runs each). attn_decode on 8-row rounds:

cache	main	branch	L3 miss rate	DRAM read per launch
int8	367.0 µs	419.2 µs (+14 %)	60.3 → 65.3 %	133–136 → 139–148 MB
bf16	491.8 µs	395.4 µs (−20 %)	69.9 → 68.5 %	194–224 → 130–139 MB

• In bf16 the new order reads about a third less from DRAM, which lines up with its gain. In int8 DRAM reads do not fall (133–136 MB against 139–148 MB; I treat the rise as noise) and the L3 miss rate goes up by 5 points.
• Achieved read bandwidth is 328–455 GB/s, 54–75 % of the card's 608 GB/s, so the memory link is not saturated.
• A 1-row launch reads about 28 MB (int8) and 30 MB (bf16). An 8-row launch reads several times that, but less than 8 times.
• The int8 cache stores half the bytes of bf16, yet a 1-row int8 launch reads nearly as much DRAM as a bf16 one. I have not explained that.
• I only have the L3 miss rate from this run, not an L2 hit rate. VTune samples counters every 1 ms against kernels of 58–490 µs, and GB/s varies by 5–15 % between runs.

The int8 path itself. Per key both paths issue the same 32 loads (16 for K, 16 for V), each one subgroup message of 16 bytes in int8 or 32 bytes in bf16, plus the two scale loads on the int8 path. In the disassembly the int8 convert is 4 operations per element against 1 for the bf16 expand, +14 % instructions per key compared with bf16. That comparison is int8 against bf16 on one binary; it does not explain the +14 % time between main and branch, where the code is the same.

Two attempts not worth repeating.

• Old order for every int8 cache (in my last comment): restores 8K, gives back the int8 gain at 32K.
• One read per slice for all rows (8 subgroups per work group sharing the K/V slice in SLM): same bits on the attention contracts and on 16 of 16 served digests, but slower everywhere (drafted 8K −1.7 to −1.9 %, serial decode −16 % at 8K and −31 % at 32K). It did not cut DRAM reads either: a 1-row launch read 127–146 MB against 25–27 MB. Dropped.

The rule from item 6, old order for an int8 cache with few slices, still fits my numbers. My sweep's crossover is between 7,588 (−4.1 %) and 15,022 (+2.5 %) tokens, about 12.2K by interpolation, or about 380 slices on the 32-key split. That is a starting point, not a measured threshold, and as you wrote, switching would need the contract to cover both orders with measurements on each side.

I'll run tf-xpu-contract-run on the new head and post the summary when the card is back on.

[Mirrdhyn]

tf-xpu-contract-run on this head (078cac06b), Arc Pro B70 (BMG-G31), built with zig build -Dxpu-contract=true -Doptimize=ReleaseFast tf-xpu-contract-run and run with --device 0. Stack: kernel 7.0.0-43, compute-runtime and ocloc 26.35.39758.21, IGC 2.41, no ocloc workaround.

The new control goes red where you expected, at 16 rows over 33 keys:

FAIL shapes control: xpu attn_decode launched on the old (slices, heads, rows) grid: at 16x6144x33 its bytes differ from upstream 1a5644892 attn_decode + attn_merge's from byte 6144
CONTROL control: xpu attn_decode launched on the old (slices, heads, rows) grid: red at shapes, as planted

Summary:

SUMMARY bmg-g31 0xe223 10 kernels: 10 pass gates 1-4, 0 fail, 0 known findings; 7 controls: 7 red as planted, 0 missed; 0 limit problems; gates 7740 ms, limits 1 ms