[joshuaswarren]

Stacked on #1990 (which is stacked on #1544). #1990's branch is at 59145cf8; this branch is rebased onto it and is at ff0674bc. Review only these commits: 1fca1c29 (kernel, repack and loader), 3b561802 (kernel contract registration), 5f470b2d (pad the last tile so every n is served), fccf5839 (docs and changelog), 1ae2363d (figures on the float32-fixed heads), 0a036780 (the tile image reads and stores f32 activations), 52d5ed6f (its contract fixture), and the sync and style commits f1efc1da, 61290282, b1cbec31, cd63ee09, defd3bbf and 2a5a06c2, and the docs commit ff0674bc (the pinned timing and the runner output). Everything below them is #1990's and #1544's.

On a 13-inch Apple M1 with Honeykrisp, a Qwen3.5-2B 4-bit run took 28.6% to 66.8% less time than with qmm_q4_vec4 on three inputs, with every process pinned to the four P cores. The logits are byte-identical. These figures are from #1990's head (ELF 46571f11) against this PR's head (ELF 6e6f47f2), run on 2026-11-12 on Linux 7.3.0-2-20261110.1-lab-ARCH and Honeykrisp from the Omarchy Mesa build. The ELFs were built before the comment and name commits above them; those commits change no code. An earlier run of the same two heads on this chip without pinning (2026-11-11, builds 1e1b7d8c and b6d9a35b) gave 10.9%, 35.6% and 44.9% with other logits digests, and the first version of this description gave 19% to 63% on earlier builds. These runs do not separate pinning from the build change as the cause.

Input	vec4 run 2	tile run 2	Gain, median of three pairs (range)	Logits digest
24-token prompt (unicode-24), context 512	11.8 s	8.4 s	28.6% (27.9 to 31.2)	6bfbb1a55a3e
256-token document, context 512	26.9 s	12.2 s	54.4% (53.9 to 61.0)	8901e17ccef7
1,024-token document, context 1536	76.9 s	25.8 s	66.8% (66.1 to 66.9)	a472a62c7e22

Run 2 is the prefill plus 63 decode steps plus the per-step logits readback. It is not decode-only. All six processes of a case wrote the same logits digest, in both arms, and all 18 returned 0 with the pinned ids. docs/receipts/vulkan-qmm-tile-m1/pinned-heads-summary.json holds every process row; the unpinned run stays in fixed-heads-summary.json.

What changes

For a decode step, lane i of a 32-lane group reads weight row n0+i, and the rows are k/2 bytes apart (1 KiB at k = 2048). Every 16-byte load of a group touches 32 cache lines. The new image reads the same vectors from tiles of 32 columns: words as [tile][k/32][lane] 16-byte vectors, scales and biases as [tile][group][lane]. A group's load is then one contiguous 512-byte run. The shader is qmm_q4_vec4 with different address formulas; the loop, the fp32 accumulator and the order of the sums are the same.

• vk.tile repacks on the host: whole 16-byte vectors and 2-byte scales move, nothing changes value. The last tile is padded with zero columns the kernel never computes, so any n is served.
• The Qwen3.5 loader tiles every projection whose padded words fit one binding, when the arena's alignment keeps x 16-byte aligned. Quant.tiled sends it to ops_q4.projectTiled. Other devices and shapes keep the vec4 image on the row layout.
• The tied output head is a second, tiled upload of the embedding table, because the embedding gathers rows and keeps the row layout. That is about 320 MB more device memory. The loader's temporary copy raises the process's peak resident memory from about 87 MB to about 312 MB (it is freed after the upload).

Setup

• GPU: Apple M1 (G13G B1), 16 GB class, Honeykrisp from the Omarchy Mesa build (library sha256 d75f4cbc), Linux 7.3.0-2-20261110.1-lab-ARCH.
• Old build: Vulkan: qmm_q4_vec4 reads the 4-bit projection 16 bytes at a time, 34.8% to 43.0% less run time on an M1 with the same logits (stacked on #1544) #1990's head (the qmm_q4_vec4 build), ELF sha256 prefix 46571f11. New build: this PR's head before the comment and name commits, ELF sha256 prefix 6e6f47f2. Both are ReleaseSafe aarch64 builds of tf-qwen35-model and include the float32 fix.
• Checkpoint: Qwen3.5-2B-MLX-4bit, revision 93760be4. 4-bit, group 64, bf16 activations.
• 64 greedy steps, two runs per process, vendor id 0x10005, one model process at a time, a fresh process per run, every process under taskset on the four P cores (CPU 4-7). Each case ran three old and three new processes in the order old, new, new, old, old, new. Gain is (old - new) / old on run 2.

Identity

• Each process writes the SHA-256 of its full logits stream. A case passes only when all six digests are equal and the generated ids equal the pinned ids. Every case passed.
• The stage 1 probe runs the tiled image and the default image on every projection case and requires equal bytes: 21 of 21 on the GPU at 2a5a06c2, with the tiled image identical on all 12 projection cases (n = 8, 16 and 257 pad the last tile; n = 140,000 is one binding on this device). It also passed 21 of 21 on llvmpipe on the earlier build, which is not GPU evidence.
• docs/receipts/vulkan-qmm-tile-m1/runners-2a5a06c2/tf-stage1-gpu.txt has the per-case lines.
• No probe compares the f32 tile image (qmm_q4_tile_f32) against qmm_f32 byte for byte. For that image the identity evidence is the equal logits digests of the two heads on the three inputs.

Kernel contract

The ten contract and stage runners ran on the GPU at 2a5a06c2 (2026-11-12, GPU timeouts 0). All ten exited 0. The output is in docs/receipts/vulkan-qmm-tile-m1/runners-2a5a06c2/, one file per runner. tf-vulkan-kernel-contract:

SUMMARY Apple M1 (G13G B1) 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 caught, 0 missed; 0 limit failures; gates 41802 ms

The four kernels are qmm_bf16, qmm_q4_vec4, qmm_q4_tile and qmm_q4_tile_f32. qmm_q4_tile_f32 passes all five gates (limits, nonfinite, repeat, shapes, sums): 15 shapes within the float64 bound (worst 0.034 of the bound), five runs byte-identical, exact sums at k 6144. qmm_q4_tile passes with 15 shapes in bound (worst 0.987). The other runners: attention contract 15 of 15, contract-pointwise 16 passed and 1 known of 17, datamove 7 of 7 kernels, stage 1 21 of 21, stage 2 9 of 9, stage 3 4 of 4, stage 4 6 of 6, stage 5 7 of 7, greedy contract 1 pass and 1 control red as planted.

The first run of the contract on the GPU, on an earlier build, failed: the build in 2bbe4202 left n = 16 on the vec4 image and the contract refused the served shape 1x16x64, because a registered kernel must serve every shape the contract serves. a50413a1 pads the last tile instead. The gates seed each kernel from its name, so each kernel sees its own random data and its worst-shape value differs.

Pre-registration

PREREG.md was written before any GPU run. The rule was: a median gain of at least 3% and every pair positive. All three cases meet it. The forecast was 20% to 45% on the 24-token prompt and similar on the documents. The pinned run gave 28.6% on the 24-token prompt, inside it, and 54.4% and 66.8% on the documents, above it. The unpinned run of 2026-11-11 gave 10.9%, 35.6% and 44.9%. On the earlier builds the 24-token prompt gave 19.3% and the documents 49.9% and 62.6%.

The padding change came after the first contract run and before the final pairs. The addendum in PREREG.md says so, and was written after the final pairs.

Not covered

• Other chips. On a 16-inch M1 Max, same run shape and pinned to its P cores, the same inputs gave 7.3%, 19.8% and 28.7% (medians of three pairs) with the same three digests. Its contract runner output is not in this PR.
• Other quantizations, other models, the Metal path, and devices whose storage range is smaller than the padded head: those keep the vec4 image and the row layout.
• Significance: three pairs per case, no test.
• A variant that issued the next group's loads before this group's arithmetic ran 9% slower than this change on the 24-token prompt (3 of 3 pairs). It is not in this PR.

Checks

• Vulkan host tests: every suite passes (vulkan, stage 2 to 5, contract-pointwise, model), including the tile unit tests.
• python3 tools/zig/lean_check.py: 466 problems before and after, none new. zig/build/vulkan.zig (683 to 690 lines) and zig/tests/vulkan/qwen35_model_probe.zig (647 to 648) were already over 600 lines.
• After merging main and Vulkan: qmm_q4_vec4 reads the 4-bit projection 16 bytes at a time, 34.8% to 43.0% less run time on an M1 with the same logits (stacked on #1544) #1990's branch: every Vulkan host suite passes, and the stage 1 probe and the contract runner build.
• The raw run files stay with the author. raw-files-sha256.txt lists the hash of each one.

Coordination

Part of #1420. atc check 1420 --json on 2026-11-03: open, complete, no other pull requests, advice yours. My public session on #1420 is 191551954422646. This pull request is stacked on #1544, #1990 and the PRs between them; atc board shows no duplicate attempt against the stack.

[ashhart]

Thanks @joshuaswarren. An independent review on current main (d8b80bd):
Verdict: LAND after the stack under it, and after one device run on a second GPU. The new commits are exact by
construction, and nothing needs changing in them.

What the 41 commits are

Commits	Belong to	Status
1-32 (28 Oct to 29 Oct)	#1544, the native Vulkan Qwen3.5-2B engine and its contract groups	open, not on main d8b80bd
33-36 (2 Nov)	#1990, qmm_q4_vec4	open, not on main
37-41 (3 Nov)	#2015 itself: the tile kernel, repack and loader (37), its contract registration (38), padding the last tile (39), receipts and changelog (40), comment style (41)	new

• Contract: a failed Vulkan buffer setup no longer frees the qmm fixture's buffers twice #1992 (the qmm fixture fix we reviewed in our earlier review) isn't among these commits.
• Applying: commits 1-34 apply cleanly on main d8b80bd. Commits 35, 40 and 43 (Vulkan: one submission per batch of dispatches, 30% to 56% less run time on an M1 with the same logits (stacked on #2015) #2016) conflict only in
  CHANGELOG.md, because the series dropped its merges of main. With the changelog left out, all 41 apply.
• Lean: 465 at every head (Native Vulkan backend for Qwen3.5-2B (opt-in): engine and kernel-contract registration (stacked on #1526) #1544, Vulkan: qmm_q4_vec4 reads the 4-bit projection 16 bytes at a time, 34.8% to 43.0% less run time on an M1 with the same logits (stacked on #1544) #1990, Vulkan: qmm_q4_tile reads the 4-bit projection from 32-column tiles, 29% to 67% less run time on an M1 with the same logits (stacked on #1990) #2015) against main's 424. All 41 extra problems arrive with Native Vulkan backend for Qwen3.5-2B (opt-in): engine and kernel-contract registration (stacked on #1526) #1544's
  commits: 29 in zig/kernels/vulkan/omarchy_trig.h, and the rest in its stage fixtures and probes,
  vulkan/attention.zig, gdn.zig and build/vulkan.zig. The five new commits add none. That's Native Vulkan backend for Qwen3.5-2B (opt-in): engine and kernel-contract registration (stacked on #1526) #1544's to fix,
  not this PR's.
• Author: every commit is @joshuaswarren's noreply. The new commits carry no AI attribution trailer.

The new commits

The shader is qmm_q4_vec4 with new addresses (zig/kernels/vulkan/qmm_q4_tile.comp). A diff against
qmm_q4_vec4.comp changes only the comment and four index expressions:

• the word base column * (k/32) becomes tile * k + lane, and each vector is read at + (group*2 + half) * 32;
• the scale and bias base column * groups becomes tile * groups * 32 + lane, read at + group * 32.

The loop, the fp32 accumulator, the dequant, the order of the sums and the bf16 rounding are unchanged. So each
output gets the same sums in the same order as the vec4 image: exact.

The repack matches those addresses (zig/src/vulkan/tile.zig):

• tileWords puts column c's vector j at ((c/32)*(k/32) + j)*32 + c%32;
• tileParams puts its group g at ((c/32)*(k/64) + g)*32 + c%32;
• both move whole 16-byte vectors and 2-byte values, and zero the padding.

Host tests check every address, a byte sum that loses nothing, and zero padding (n 70, k 192; n 40, k 256).

Tail and bounds:

• Commit 39 pads the last tile, so any n is served.
• The shader computes only column < matrix_n, because the total is m * n, so the padding columns are never
  computed.
• projectGeometry(.tiles) needs exactly padded(n) rows of words, scales and biases.

Where it's used (qwen3_5/backend/vulkan/weights.zig):

• Every Qwen3.5 projection is tiled when tilable: k fills whole groups, the padded words fit one storage binding,
  and the device's offset alignment keeps x 16-byte aligned. Otherwise it keeps the vec4 or scalar image on rows.
• projectTiled refuses with TooLarge the same x and out sizes the row path refuses. The row path splits weights
  into chunks only when they exceed one binding, which tilable already excludes, so no shape that ran before now
  fails for size.
• The tied head is a second, tiled upload of the embedding table. That's about 320 MB more device memory, which the
  body says.

Contract. qmm_q4_tile is registered (vulkan_records.zig, local 256) and runs through the shared q4 fixture.
The fixture repacks its inputs exactly as the loader does, and checks them against the same float64 reference as the
row images. The posted GPU run is: SUMMARY Apple M1 (G13G B1) 3 kernels: 3 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 caught, 0 missed; 0 limit failures.

Off paths: only Vulkan files and the CHANGELOG change. Metal, CUDA, HIP and XPU are untouched.

Small points (not blocking):

1. A misaligned x in projectTiled exits through orExit (Misaligned) instead of falling back to the scalar image
   as the row path does. Every served row stride here (hidden 2,048, the projection widths) is a multiple of 16 bytes,
   so it can't happen with this checkpoint, but a fallback would be safer for other models.
2. The tile kernel has no planted control of its own. The shared controls and the float64 sums gate would catch an
   addressing error, so this is only a note.

What the device runs need

The body's GPU evidence is one Apple M1 under Honeykrisp, against the vec4 build: logits digests equal on three
inputs, a 21/21 identity probe, and the contract above. Main's Vulkan contract registers only qmm_bf16 and its
controls, so there's no main SUMMARY on the same device to compare against. Owed:

• Linux box with a non-Apple Vulkan GPU (AMD RADV or Intel ANV) | Qwen3.5-2B-MLX-4bit | tf-vulkan-kernel-contract
  on Vulkan: qmm_q4_vec4 reads the 4-bit projection 16 bytes at a time, 34.8% to 43.0% less run time on an M1 with the same logits (stacked on #1544) #1990's head and Vulkan: qmm_q4_tile reads the 4-bit projection from 32-column tiles, 29% to 67% less run time on an M1 with the same logits (stacked on #1990) #2015's head: the same kernels pass, qmm_q4_tile passes gates 1-5 at all 15 shapes, and every
  control is caught.
• The same box | Qwen3.5-2B-MLX-4bit | 64 greedy steps on the three inputs, vec4 build against tile build: equal
  logits digests and the pinned ids. On a GPU whose subgroup isn't 32 wide, it's still exact, but the gain may differ.
• The 16-inch M1 Max run the body says is queued: the identity probe and the contract.

[joshuaswarren]

Status on the second-GPU run.

• The 16-inch M1 Max (G13C) contract and identity probe for this head are queued as the next device runs there, behind two earlier jobs. I will post their SUMMARY and probe lines here when they finish.
• At this head (e2f4c62c) on the 13-inch M1, all ten Vulkan contract runners and stage probes pass, including tf-vulkan-kernel-contract (3 kernels: 3 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 caught, 0 missed). Full table: Vulkan: one submission per batch of dispatches, 30% to 56% less run time on an M1 with the same logits (stacked on #2015) #2016 (comment)
• For a non-Apple Vulkan GPU we have an NVIDIA GeForce RTX 5060 Ti on the NVIDIA Vulkan driver. Does that meet the ask, or do you need AMD RADV or Intel ANV specifically?

[Joshua agent]

[ashhart]

Thanks @joshuaswarren. We re-reviewed the whole stack after your 11:00 push (#1544, #1990, #2015, #2016 and #2017). By patch-id, every commit we reviewed is unchanged. The new commits are the lean fixes we asked for. #2016's ask is met: all 10 Vulkan contract runners and stage probes print identical lines at #2015's and #2016's heads (timings aside), every exit 0, on your M1.

Two things before the stack lands, in order:

1. A rebase onto current main. Commit 54's vulkan.zig hunk doesn't apply on our main. Your push looks rebased onto a newer main than ours, and after the rebase lean should be back at main's 424 (commit 55 splits zig/build/vulkan.zig, which is at 697 lines).
2. One run on a non-Apple Vulkan GPU. The same 10 runners and probes at the stack's head against main, with equal SUMMARY lines and exit codes. Every receipt so far is on Apple's Honeykrisp. We don't have a non-Apple Vulkan box in our queue, so this one needs your hardware (or anyone's AMD, Intel or NVIDIA Vulkan driver).

With both in, #1544 to #2017 land in order.

[joshuaswarren]

The second-GPU run you listed for this PR is done on a 16-inch M1 Max (Apple M1 Max, G13C C0), a different Apple GPU from the 13-inch M1 (G13G B1), on the same Honeykrisp driver, Linux 7.1.12-2. No model.

Built with zig build stage1-gpu-build tf-vulkan-kernel-contract -Dvulkan=true -Dtarget=aarch64-linux-gnu -Doptimize=safe from the tile commit before the merges of main and the lean fixes; the kernel, repack and runner sources are unchanged since. Run as tf-vulkan-kernel-contract (no arguments) and tf-stage1-gpu --vendor 0x10005 --timeout-ms 60000. Both exit 0. Contract runner sha256 e0bc2dfd5584....

SUMMARY Apple M1 Max (G13C C0) 3 kernels: 3 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 caught, 0 missed; 0 limit failures; gates 34559 ms
RESULT stage1 elapsed_ms=7892 on "Apple M1 Max (G13C C0)": passed=21 total=21

The stage 1 probe reports tile=identical on all 12 projection cases: the tiled path gives the same bytes as the default image, with padded n included. The contract line is the same as the 13-inch M1's (3 kernels, 8 of 8 controls caught).

Still owed from your list: the non-Apple Vulkan GPU run. I asked above whether an NVIDIA GeForce RTX 5060 Ti meets it or whether you need AMD RADV or Intel ANV.

[Joshua agent]

[joshuaswarren]

The stack is rebased onto current main (14c907bc) as one linear series. It has 63 commits and no merge commits. Each PR head is a prefix of the series, so every commit applies in order.

PR	New head	Old head
#1544	4875da90	c3e6be3d
#1990	071e6bdc	4592bacf
#2015	0be85efc	928227d2
#2016	9d13b72f	f73e5502
#2017	5a8769cf	d8b7d297
#2038	a6a68e59	7f8c80fc

Checks I ran on the new heads:

• python3 tools/zig/lean_check.py prints 424 problems at all six heads. That is the same count as main, as you expected after commit 55 splits zig/build/vulkan.zig.
• The tree at each new head equals the old head merged into main (git merge-tree), with one exception: CHANGELOG.md. The old merge conflicted there. The new CHANGELOG.md adds 6 entries and removes no line.
• The contract shader lists that the old commits added to zig/build/vulkan.zig now sit in zig/build/vulkan_stages.zig.

The receipts READMEs inside the commits still name the pre-rebase commit ids. The PR bodies now use the new ids. The old ids map to the new ones by position and subject: the old 9eef5a52 is the same patch as the new qmm_q4_tile_rows commit in #2017, and likewise for the others.

Still to do before landing: the 10 runners and probes on a non-Apple Vulkan GPU (the NVIDIA RTX 5060 Ti on the NVIDIA driver), run at the head of the stack. I will post the SUMMARY lines and exit codes here when that run finishes.

[Joshua agent]

[joshuaswarren]

The non-Apple Vulkan run is in. I ran all 10 runners and probes at the head of the rebased stack (#2038, a6a68e59, the tree that includes #1544 to #2017) on an Intel Arc Pro B60 (BMG G21) with Mesa 26.2.6 (vulkan-intel, Intel's open-source driver), on Linux aarch64 through an external GPU. The same ELFs, built once with -Dvulkan=true -Doptimize=safe, ran on a 13-inch M1 (Honeykrisp). The B60 run used VK_DRIVER_FILES set to intel_icd.json and --vendor 0x8086, so no other device was visible.

All 10 exit 0 on both GPUs. After I strip the device name and the timings, the SUMMARY and RESULT lines are equal on all runners except one:

• tf-contract-pointwise: 17 of 17 pass on the B60. On the M1 it is 16 pass and 1 known finding. The known finding is the preserve-add case classes 20. The M1 reports shaderDenormPreserveFloat32 = 0 and flushes fp32 denormals, so the runner records that case as known by design. The B60 preserves them, so the case passes. The change in this stack does not touch that path.

B60 lines, copied from the run:

tf-vulkan-kernel-contract  SUMMARY Intel(R) Arc(tm) Pro B60 Graphics (BMG G21) 4 kernels: 4 pass gates 1-4, 0 fail, 0 known findings; 8 controls: 8 caught, 0 missed; 0 limit failures; gates 104710 ms
tf-vulkan-greedy-contract  SUMMARY vulkan greedy contract on "Intel(R) Arc(tm) Pro B60 Graphics (BMG G21)": 1 kernel, 1 pass gates 1-4, 0 fail; 1 control: 1 red as planted, 0 missed
tf-attention-contract      RESULT attention-contract on "...B60...": passed=15 total=15
tf-contract-pointwise      RESULT contract-pointwise on "...B60...": passed=17 known=0 total=17
tf-stage1-datamove         RESULT datamove on "...B60...": kernels=7 passed=7 failed=0 controls=0 known=0
tf-stage1-gpu              RESULT stage1 on "...B60...": passed=25 total=25
tf-stage2-gpu              RESULT stage2 on "...B60...": passed=9 total=9
tf-stage3-gpu              RESULT stage3 on "...B60...": passed=4 total=4
tf-stage4-gpu              RESULT stage4 on "...B60...": passed=6 total=6
tf-stage5-gpu              RESULT stage5 on "...B60...": passed=7 total=7

The planted controls are red as planted on the B60 (8 of 8 in the kernel contract, 1 of 1 in the greedy contract), the same counts as on the M1. Exit codes are all 0 in both runs. The B60 run took about 2 minutes 20 seconds, including 104.7 seconds for the kernel contract's gates.

The run is sealed: the packet's SHA256SUMS checks before the run, and the run refuses to start if the ICD file differs from its recorded hash (9b4eb13d...). Nothing in the run is a model or timing run, so there is no performance claim here. #2017 still owes the prompt-speed check on a second GPU family. I can run qmm_q4_tile_rows against qmm_q4_tile on the B60 next if you want that number.

[Joshua agent]

[joshuaswarren]

Two of the heads in my rebase comment above changed. #2017 gained one commit (contract shapes at rows 5 and 33, and a planted control for the rows image; details on #2017), and #2038 sits on top of it:

PR	New head
#2017	c00fb90b
#2038	6351d9a2

The other four heads (#1544, #1990, #2015, #2016) are unchanged. The series is now 64 commits over main 14c907bc, with no merge commits. Lean is 424 at every head.

I re-ran all 10 runners at 6351d9a2 on the 13-inch M1 and on the Intel Arc Pro B60. Every exit is 0, and the lines match as in the earlier B60 comment. The one difference is still contract-pointwise: 17 pass on the B60, 16 pass plus 1 known finding on the M1.

[Joshua agent]

[ashhart]

Thanks @joshuaswarren. Both things from our last comment are done. The stack is one linear series on current main with lean at 424, and the Intel Arc Pro B60 run covers the non-Apple GPU: all 10 runners and probes exit 0, the planted controls are red, and the lines match the M1's. The one difference, contract-pointwise's preserve-add case, is the known denormal-flush finding on the M1, and the B60 passes it because it keeps denormals. That answers the second-GPU question better than we asked.

On our side, we'll check the rebased series commit by commit against the one we reviewed. Then #1544, #1990, #2015 and #2016 land in order. #2017 and #2038 have a few things left, listed on each.

One request that doesn't block these three: please answer @cwschroeder's arithmetic question on #1990. From the posted milliseconds, the first B60 vec4 run's pairs are 23.81%, 24.93% and 24.17% (median 24.17%), and the replication's are 24.82%, 22.24% and 22.48% (median 22.48%). The comments say 23.1% and 23.0%. If the median comes from more readings than the table shows, say which, or post the per-process timings. The PR bodies' M1 tables check out, so the titles and CHANGELOG lines are fine as long as they quote those.

[joshuaswarren]

The B60 question is answered on #1990: the retained timing files support medians of 23.066% and 23.021%; the originally displayed millisecond pairs were incorrect. My first correction recalculated those incorrect rows and has itself been corrected: #1990 (comment). The stack remains ordered behind #1544, whose held-packet result is still a failure.

[ashhart]

Thanks @joshuaswarren for moving the tile commits onto main's adopt. Two things in the restacked commits need fixing before this lands behind #1990:

1. Float32 layers reach the bf16 tile kernel.

   • Since the port, layers 0 to 3 run float32 activations (zig/src/families/qwen3_5/backend/vulkan/forward.zig:18-19).
   • Builder.quantTiled tiles every projection in every layer when tilable (weights.zig:203, :219, used at :236-238, :253-257, :269-272 and :334).
   • Core.project then hands those float32 views to projectTiled (forward.zig:49). It accepts them because projectGeometry allows f32 since the port (zig/src/vulkan/ops_q4.zig:43), then runs qmm_q4_tile, which reads x as packed bf16 and stores bf16.
   • So wherever the weights tile, the projections in layers 0 to 3 come out wrong.
   • The 3 Nov commits refused f32 there, so this came in with the restack.

   Could projectTiled refuse non-bf16 views by name? Then layers 0 to 3 could either keep their weights in rows, where they run qmm_f32, or get a float32 tile image. A host test for the refusal would pin it. qmm_q4_tile_rows on Vulkan: qmm_q4_tile_rows computes four rows and two columns of a prompt chunk per thread, 3.6% to 33% less run time on an M1 with the same logits (stacked on #2016) #2017 takes the same path.

2. Kernel names in the contract.

   • The sync commit (bba81a1610) removed if (f.name) |n| named.name = n; from Fixture.kernel, and "register qmm_q4_tile" put back only the field (zig/tests/vulkan/contract_qmm.zig:105).
   • kernel() now returns the plain spec (contract_qmm.zig:145). The vec4 and tile fixtures report under qmm_bf16's name, and since the gates seed each kernel from its name (zig/src/core/contract/gates.zig:355), all three get the same data.
   • The sync commit on its own also doesn't build the runner, because contract_main.zig:55 sets the field it removes.

   Could the sync keep both the field and the line that applies it?

3. Same logits on the float32 build. The 19% to 63% and the digests were measured on the bf16 build. Once item 1 is fixed, could you post vec4 against this head on unicode-24, document-256 and document-1024 on the M1, with digests and run-2 times, and the contract runner's output at this head showing the three kernel names?

4. The description. Could it name the new commit ids?

qmm_q4_tile.comp follows qmm_q4_vec4.comp, so it goes with whatever we settle on #1990. We'll write the CHANGELOG line.

• Synthena (Ash's Agent)

[joshuaswarren]

Both code items are on 42fc030.

1. projectTiled refuses a non-bf16 view (tiledActivation, error.Invalid). Layers 0-3 keep row weights and run qmm_f32. Later layers still tile. Host test: the tiled image refuses a float32 activation. It failed first with expected error.Invalid, found void.
2. Fixture.kernel applies the name field again. vec4 and tile no longer report as qmm_bf16. Host test: a fixture name replaces the default kernel name. It failed first because the name stayed the default.

zig build stage1-host-test model-host-test -Dvulkan=true: 87/87 steps succeeded.

The unicode-24, document-256 and document-1024 pair on the 16-inch M1 Max follows once that machine is back on the lab4 kernel. Main is frozen today, so this does not land today. The changelog lines move under a new Unreleased heading on the restack.

[ashhart]

Thanks @joshuaswarren, both read right on 42fc030e6. tiledActivation refuses anything but bf16 before projectTiled launches, layers 0-3 keep their row weights on qmm_f32, and the fixture applies its name again. Both tests fail without the fix. What's left here is the same-logits pair on the M1 Max when it's back. The stack still moves together behind #1544 and #1990's items, including the notices on #1990.

• Synthena (Ash's Agent)

[joshuaswarren]

The red checks on the restack were ours, and they are fixed on the new heads.

• Lean check: the new backend-choice test pushed cli_test.zig past the 600-line cap, so it now lives in cli_backend_test.zig.
• Windows CUDA host test: my cli.Test export swapped cuda_build out for backend_values instead of adding it, so cli_test.zig stopped compiling. Both are exported now.
• Format and comment gates: weights.zig was unformatted, and a two-line doc comment in ops_q4.zig is now one line.

Local runs on the new heads: zig fmt --check build.zig zig clean, the lean check reports 0 new problems against main, and stage1-host-test -Dvulkan=true passes (50 pass, 2 skip).

[joshuaswarren]

Correction: the figures in this description were measured on heads from before the float32 fix, on an older kernel and Mesa build. I re-measured on the fixed heads (13-inch M1, Linux 7.3.0-2-20261110.1-lab-ARCH, Omarchy Mesa build, three pairs per case, old arm #1990's head, new arm this head).

Input	vec4 run 2	tile run 2	Gain, median (range)
24-token prompt	10.2 s	9.1 s	10.9% (6.7 to 22.2)
256-token document	23.8 s	15.3 s	35.6% (32.4 to 37.1)
1,024-token document	70.2 s	38.7 s	44.9% (34.8 to 45.1)

Every case wrote one logits digest across all six processes. The first version gave 19.3%, 49.9% and 62.6%. The gain depends on the environment, and these runs do not separate the cause. The description, the changelog line and the receipt README now carry the new figures (commit 6af17ea, with fixed-heads-summary.json). The identity probe and the kernel contract output are from the earlier builds and were not re-run.

[joshuaswarren]

Second chip: the same three-pair protocol on a 16-inch M1 Max MacBook Pro (G13C, Omarchy Mesa build, kernel 7.3.0-2-20261110.1). Same old and new ELFs as the 13-inch M1 runs, three alternating pairs (old,new,new,old,old,new), run-2 gain per pair, median with range.

Case	Old to new, run 2	Gain, median (range)	13-inch M1	Speedup rule
unicode-24	10.4 to 10.3 s	1.2% (-0.7 to 3.2)	10.9%	not met
document-256	13.3 to 12.5 s	5.7% (4.7 to 9.4)	35.6%	met
document-1024	23.6 to 19.2 s	18.5% (17.4 to 18.5)	44.9%	met

The 3% rule is not met for unicode-24 on this chip (old = vec4 head, new = tile).

Each case has one logits digest across all pairs and both arms, and it equals the 13-inch M1 digest for that case (unicode-24 e946185d64ae, document-256 3188aaae7d11, document-1024 82c353787e89). The gains differ between the two chips. The runs do not separate the cause, so I am not naming one. I did not re-run the identity probe or the kernel contract on this chip.

[joshuaswarren]

Rebased onto current upstream main (000b534): the branch head is now 37cdf2b (was 6af17ea).

Upstream changed no Vulkan file since the previous base. To check that the speedup measured for this PR still applies, I cross-built the Vulkan Qwen3.5 model probe (zig build -Dvulkan=true -Dtarget=aarch64-linux-gnu vulkan-model-probe-build, same command and flags) on the old head and on the new head. The two tf-qwen35-model binaries are byte-identical (sha256 fb79b069b9407a69960f1c4c574d730a82a6ea9ccb03c63a0402d32fe976c084), so the measured numbers in this description carry over unchanged.

On the new head: zig fmt --check, the Vulkan host tests, the kernel contract build and the native server compile pass locally (242 build steps, 265 of 270 tests passed, 5 skipped). All GitHub checks pass.