[cshintov]

This builds Qwen3.6-35B-A3B (text, no MTP, no images) on main's routed expert path (#1125), as asked when #1009 closed. Slice 1 of #931.

What this changes

• Admits the 35B-A3B on the routed path (qwen27: Qwen3.6-35B-A3B on #1125's routed path...). model.supported takes its shape, and the attention Geometry takes 16 query heads over 2 (a group of 8) on one Mac. A rank of 8 over 1 is refused. Config tests cover the shape and both refusals.
• The routed block's contract cases at the 35B's served widths: 512 experts' rows over a hidden width of 2,048.
• qwen27 attention's decode and prompt row cases at 16 over 2, beside main's 24 over 4 and 32 over 2.
• docs/kernel-contract.md: one paragraph on the new widths.
• Staged references (tf-qwen27-run --stages DIR). It writes the residual stream after the embedding, after each layer's mixer and MLP, then final and the logits.
  • final is the normed rows the head reads, not the residual. A fixture with a non-unit final norm refuses a residual labelled final.
  • tools/zig/qwen36_stage_ref.py measures each stage against a float64 forward of the checkpoint's exact weights. It reports the local error (each stage fed the engine's own input) and the chained error (from the token ids).
  • A headless prompt chunk records 0 logit rows. While stages record, the last layer is treated as tapped, so main's keys-only last layer stays off: the recorded stream and final are real rows.
  • Off by default. A round that isn't one stream's chain is refused before it touches any state.
• Lean: comments on one line within 120 columns.

Design

The 35B-A3B is the same family as Ornith-1.5-122B with other widths: 256 experts of 512 and a shared expert of 512 over a hidden width of 2,048, and 16 query heads over 2. So it adds no kernels and no loader. The router runs on its stored 8-bit g64 rows, stacked with the shared gate.

Alternatives rejected:

• Qwen3.6-35B-A3B on Metal: routed experts in the 27B's engine (slice 1 of #931) #1009's own router and expert kernels: duplicate paths that main's routed path already qualifies.
• A second family directory: main keeps routed checkpoints in the qwen27 engine, and this follows it.
• Staged references through the forward pass's single hook pointer: the hook now carries its own planes and glue, so the runner needs no forwarding function.
• Recording stages beside the keys-only last layer: its rows aren't written, so the recorder would read stale bytes.

Receipts

M4 Pro (no tensor units), macOS 26.5, head cc5048b6c on main aaac299d1.

Host and contract:

• zig build test-qwen27: all pass (46 of 46 steps).
• zig build test-kernel-contract: 204 of 204 pass.
• tf-kernel-contract --only qwen27, on the same commits over main c34ff562c: 36 kernels pass gates 1-4, 0 fail, 0 known findings; 3 controls red as planted; 0 limit problems.
• zig fmt --check: clean. tools/pr_check.py --base main: 0 errors, lean 341 against 341 (0 new).

Real checkpoint (mlx-community/Qwen3.6-35B-A3B-4bit at 38740b84), on this head. tf-qwen27-run --max-tokens 64 --chunk 128 --compare-chunk 16 --seams 16, greedy:

Prompt	Prompt tokens	Decoded	Prompt state	Decode state	Seams
short	30	64	byte-equal	byte-equal	1/1 equal
code	34	64	byte-equal	byte-equal	2/2 equal
long	492	64	byte-equal	byte-equal	30/30 equal

All three replies are token for token the same on this head, on the same commits over main c34ff562c, and before the rebase. The staged references below were measured on the pre-rebase run.

Staged references for the long prompt plus 72 teacher rows, against a float64 forward of the same weights:

• Local error per stage (median of the means over stages): Metal 0.25%, mlx_lm 0.31%.
• final local max: 0.19%.
• The worst local stage is one row each, where the float64 router gap is under 0.015 and bf16 routes to another expert: Metal 15.6% at layer 12, mlx_lm 17.8% at layer 13.
• The chained error grows alike in both engines, to a median of 42% by layer 39, from bf16 rounding and routing flips.
• Top-1 against float64: Metal 72/72, mlx_lm 71/72.

Not run, and why

• Drafts on against draft: false. This engine's Metal drafter reads DFlash2 contracts only (the 27B's and Ornith's). The 35B-A3B's only published drafter is DFlash v1 (sliding-window layers, no selector), so there is no drafted Metal run to compare. A drafter for it is a separate PR.
• The M5 cases. The prompt and prefix kernels on tensor units, and the affine gathers at the 35B's widths: their records pass here, but their gates run only on tensor units.
• The full zig build test. It was not run on this host; the qwen27 and kernel-contract suites above cover the changed files.
• A dense 27B A/B. The 27B's paths change only where staged references are asked for, which is off by default.

[ashhart]

Thanks @cshintov. One build fix remains after the rebase.

At stage_planes.zig:6 and qwen27.zig:52, the code references core.qwen35_stages, but the production tensorfold.zig core does not export it. The host geometry check fails with "root source file struct 'tensorfold' has no member named 'qwen35_stages'". The standalone tools use separate core shims, so they miss this caller.

Please export the recorder through the production core and rerun zig build native test-qwen-dense-geometry-host. Once that passes, we will run the device check on our side for staged reference equality and served tokens.

• Synthena (Ash's Agent)

[cshintov]

Exported qwen35_stages from the production core (tensorfold.zig), next to qwen35. At 742ebb4 (rebased onto main 4ddc28c), zig build native test-qwen-dense-geometry-host passes (72 of 72 steps, 111 of 111 tests) on an M4 Pro, and zig build test-qwen27 still passes (180 of 180). Lean: 341 against 341, 0 new.