Repository navigation
Conversation
|
Thank you, @akol1, for the Metal family, the fp64 reference tests in #826, and an Engram token map built in Zig that matches the release byte for byte (#824). Your split on #635 is the plan. We've opened #847 to coordinate it: one package in What #824 to #827 need before they land:
This is careful work, and we want it in. Thank you again. |
|
The whole-model gate question now has a complete experimental answer, and it changes what "passing" can mean. What was measured. After the sparse-attention accumulation fix (the per-tile compensated pairs, now in the branch), the 8-token oracle dropped from 4.096% to 3.0517% NRMS with greedy agreement intact. Per-token, the error is 0.31%, 0.25%, 0.71% for the first three tokens and climbs to 5.9% by token 7 — the signature of state carried forward, not a per-call kernel error. A 12-layer same-input sweep, the MoE capture (110 comparisons) and the hyperconnection capture then came back cell-for-cell identical between the Metal chain and the reference chain; the route-region diff showed only two routing weights differing by one FP32 ULP with identical expert selections, and a decode of the cache layout confirmed our packed rows hold the same values the reference's quantization produces. What that rules out. With every captured stage bit-exact, no kernel is "wrong" in any way the operator tests can express — the residual 3.05% is the composition itself: the model's own sensitivity to FP32-level differences, accumulating through 40 layers exactly as the 2.90% FP32-vs-FP64 floor predicted. Three further experiments confirmed the direction: hardening the Engram gate's sums to wide-precision pairs, hardening the RMS reduction the same way, and switching the routing-weight sum's association each moved the number up (3.45%, 6.50%, 5.34% on a dirty run) — every deviation from the shipped arithmetic made the match worse, because the shipped kernels and the reference chain were tuned to each other. What this means for the gate. The 1% whole-model limit cannot be met by kernel work: the remaining delta is the model's FP32 sensitivity, the same term the 2.90% floor measures, and it compounds with depth. Three options, in our order of preference:
All three keep a hard correctness floor: greedy agreement 8/8 (measured on every run), the per-operator bit-exactness suite (green), and the residency gate. What we cannot do is tune kernels toward a 1% whole-model number that the model's own FP32 sensitivity forbids — the experiments above are the proof. If the gate-form decision waits, the pieces are complete as they stand: #824's golden test, #825's host tests, #826's per-operator bit-exactness including the new PV pairs, and #827's served-parity legs. The whole-model receipts for everything above are on the branch (oracle-postfix, the 12-layer sweep, the MoE/HC captures, the per-token tables). |
|
@ashhart — the two decisions that gate this series are now consolidated with explicit options on the tracker: the whole-model gate form (three candidates, the evidence above, our pick = the floor-relative bound) and the consolidation sequencing for the four shared files. Linking here so the answer lands next to the measurements: #847 (comment) |
|
Thank you, @akol1, the pinned whole-model result and precision-floor experiment keep the unresolved choices concrete. This stays with the one DeepSeek family, after shared infrastructure. We will also repair the checked verify/partial-keep transaction ownership before enabling real drafted serving. Please answer both questions on #847: which proposed whole-model gate keeps or replaces the 1% NRMS bound, with exact thresholds and long-token greedy controls; and which Engram storage/scales both loaders should read, using pull-time conversion or a pinned prepacked format. We will keep the two answers together on that tracker. The measured 3.0517% and current draft/image/state gaps remain visible until that gate is settled. Posted by Ash's AI Assistant Synthena |
cfd6f8c to
45e8c06
Compare
Require TF_EXPERIMENTAL=deepseek while whole-model gate #827 is pending. Check both checkpoint spellings at the native boundary, the dedicated server and the whole-model factory before device or payload work. Keep the existing family implementation and shared infrastructure, and label the release notes and known issue accordingly.
…ests The generated nfkd_data.zig now comes from tools/zig/nfkd_tables.py, a peer of tools/zig/unicode_tables.py: it rebuilds the table from unicodedata 16.0.0 and zig fmt, and its --check fails on drift. The table is one file with a generated header, and lean_check.py's generator-output exemption covers it, as it already covers the Flash Next generator outputs. The normalizer chain and the PCG64 draws keep their tests; the Hangul case expected a recomposed syllable, NFKD decomposes it and the release's chain never recomposes, so the expectation and a comment say so. The program was registered only as an installable executable, so its test blocks never compiled under zig build test, which is how a wrong Hangul expectation survived. The program table now takes host_tests, and the test step depends on the program's test artifact.
operator_csa2_fp64.zig (1331 lines) and operator_hc_fp64.zig (1095) sit above the 600-line file budget. Split by what the harness measures, each part with the harness helpers it uses: csa2 into the shared view/buffers, the code tables, the sparse stages, rope and the indexer; hc into the shared view/config, the split-and-recombine bit-identity case and the amplify stages. The reference root imports every part in dependency order, so the tests.deepseek_v41_reference. filter and the suite counts are unchanged. No tolerance, assertion or receipt text moved.
…e line each Folds the remaining comment groups into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
…ment `lanes` sat in the six-or-seven argument list, so the eight-argument form with CHUNK and CONTEXT was rejected before the parser could read it, and the usage text never listed the mode at all. Remove it from that list, document the mode's chunk and context arguments, and cover the accepted and rejected arities with a host test now that the runner registers its tests.
beginSpan gained a metadata argument, so the four call sites in the Metal counter consumer no longer compiled. zig build test does not build that consumer, so nothing caught it. The loop passes its index, matching the profiler convention, and the three lifecycle tests pass the field default.
1.1.2 replaced the nested text_config with one flat Config and renamed its fields, so every reader in the Metal line moves to the new names: rms_norm_eps to eps, rope_theta and the scaled rope settings to rope.*, and the layer-id, vocab and width fields to their short names. The engram table directory is a fixed-capacity byte list now, so the table plan takes its items and treats an empty list as absent. metal_targets registers the operator qualification steps beside the other families' so the device, weights and reference suites build and run again.
… picks Two things in the reference checks needed fixing: - operator_csa2_fp64.zig printed its Metal-vs-FP64 comparison but never asserted it, so any regression in pool reductions passed silently. Turn each case into a PoolDelta carrying the Metal NRMS, the FP64 reference NRMS and the Metal-vs-reference NRMS, run six cases (typical, wide dynamic range, cancellation, gate skew, balanced cancellation, and the 1e7 cancellation case), and assert each stays under the contract floor and that Metal is not materially worse than the FP64 reference. - operator_csa2_index_fp64.zig derived its overlap count from its own reference mask, so 512 copies of one valid key would have counted as full coverage. Sort the reference picks, build the chosen set from that, and count repeats and kernel-only picks over the kernel's own output; assert zero out-of-range picks, zero repeated picks, zero kernel-only picks, and that any weight flip sits within two quanta of the boundary.
…re its own tests
test-dsv41-shared did not build: the family is a module rooted at its own
directory, so its relative `../../core/*` imports reached outside the module
path, and where they resolved they gave the same file a second identity
next to the one `@import("core")` already created (names.zig used both).
Reach core by name. zig/dsv41_core_family.zig is that name's surface: it
re-exports exactly the ten core modules the family uses, and both build
lanes point their `core` at it. The family's fourteen relative up-imports
become `@import("core").<m>`, matching what upstream already does in
names.zig and elsewhere. The lanes thread that module through weights() and
checks(), and dsv41_shared.zig gives it `metal` so gpu_counters resolves.
The cost is that a test block reached through a named module never runs, so
core's files cannot be path-owned and module-owned in the same binary. Give
core one place to run instead: test-core, rooted at zig/src/core/root.zig,
with the six core test files that were only reached transitively now named
there. Every test the Metal lane used to collect by path still runs --
gpu_counters, expert_groups, safetensors and the rest in test-core,
exl3_projection_gpu_test in test-exl3-gpu -- and csa2's check moves from its
own file to kv_test.zig so it stays in the same binary.
The previous head's push did not produce a pull-request synchronize event, so no workflow ran against the CSA2 assertion fixes or the core module change. This empty commit exists only to trigger them; it changes no files.
fe02e95 to
e944db7
Compare
ashhart
left a comment
There was a problem hiding this comment.
Thanks @akol1. This replaces main's shared kv_state.zig, model.zig and load.zig with Metal-only versions, so main's CUDA DeepSeek family stops compiling (cuda_transaction.zig: kv_state has no Region). Please carry the new pieces (the fp64 references, the lane backend) onto main's shared modules instead, as one stack with #825 (see the plan there). Building a missing token map at load is a separate behaviour change; please split it out.
|
Review against main e9b69ed (full notes on #825). This PR replaces main's shared |
Correct the native support, rank and precision descriptions while keeping the frozen Python tables and measurements explicitly historical. Distinguish released images, kernel checks and checkpoint qualification. No implementation, test, lean baseline or measured speed changes. Claim audit against the main source: 1. Native CUDA registers more than Nemotron and Flash Next; registration is separate from device and checkpoint qualification. Docs: docs/recipes/cuda.md:11. Evidence: zig/src/native/cuda.zig:33, zig/src/native/cuda.zig:199, zig/src/native/cuda.zig:202. 2. Qwen4 is preparation with synthetic device gates and no real-checkpoint qualification; DeepSeek is refused pending gate #827. Docs: docs/recipes/cuda.md:13, RUNBOOK.md:201. Evidence: zig/src/native/cuda.zig:43, CHANGELOG.md:773, zig/src/native/cuda_refusal.zig:5. 3. Native GLM has recorded TP=2 and TP=3 serving, including FP4-Local and XSFP4; the CUDA setup is not limited to two ranks. Docs: docs/recipes/cuda.md:17, RUNBOOK.md:193, README.md:207. Evidence: zig/src/native/cuda.zig:33, zig/src/families/glm/cuda_native.zig:128, docs/receipts/glm-fp4local-80517fc3/README.md:27, CHANGELOG.md:517. 4. GLM uses TF_GLM_RANK, TF_GLM_WORLD and NCCL_COMM_ID; other families accept --tp only when their options consume ranks. Super VL has one- and two-GB10 receipts. Docs: docs/recipes/cuda.md:162, RUNBOOK.md:194. Evidence: zig/src/families/glm/cuda_native.zig:118, zig/src/families/glm/cuda_native.zig:180, zig/src/native/cuda_options.zig:12, zig/src/native/cuda_refusal.zig:55, docs/recipes/nemotron-3.5-super-vl.md:4. 5. EXL3 loading and DFlash2 drafting are native capabilities for particular families and formats, not Python-only features. Docs: docs/recipes/cuda.md:18. Evidence: zig/src/families/glm/cuda_exl3.zig:27, zig/src/families/glm/cuda_native.zig:291, zig/src/families/qwen3_5/cuda_native.zig:34, CHANGELOG.md:553. 6. The Nemotron MLX 4-bit serving record names GB10, RTX 3090 and RTX PRO 6000; an RTX 3090 receipt does not measure every sm_86 card. Docs: docs/recipes/cuda.md:20, docs/recipes/README.md:34, RUNBOOK.md:190. Evidence: zig/src/native/cuda.zig:33, zig/src/families/nemotron/cuda_native.zig:31, CHANGELOG.md:1662, CHANGELOG.md:1623, CHANGELOG.md:1538. 7. Release targets are sm_80, sm_86, sm_120 and sm_121. The A100 has recorded kernel checks but no served checkpoint on record, so it is not model-qualified. Docs: docs/recipes/cuda.md:22, docs/recipes/README.md:35, RUNBOOK.md:197, README.md:164. Evidence: zig/build/cuda.zig:542, CHANGELOG.md:1322. 8. Nemotron and dense Qwen have NVFP4/FP8 non-cluster fallback implementations for sm_80 to sm_89; no such model run on sm_80 is recorded. Docs: docs/recipes/cuda.md:24, docs/recipes/README.md:41, RUNBOOK.md:199, README.md:166, README.md:179. Evidence: zig/src/families/nemotron/cuda_kernels.zig:193, zig/src/families/qwen3_5/cuda_modelopt.zig:216, zig/src/cuda/qmmf.zig:153, CHANGELOG.md:1237, CHANGELOG.md:577. 9. Flash Next MLX 4-bit has a GB10 serving receipt; the ModelOpt/two-local-rank path is experimental, with tensor-rank drafting requiring the prompt cache off. There is no blanket all-SM qualification. Docs: docs/recipes/README.md:42. Evidence: zig/src/native/cuda.zig:33, zig/src/families/flashnext/cuda_lanes.zig:29, zig/src/native/cuda_refusal.zig:57, CHANGELOG.md:1221. 10. Gemma 4 12B and 31B MLX checkpoints, including assistant/DFlash drafting, have native CUDA GB10 receipts; pre-M5 Metal has a row path, while its assistant and DFlash drafters require tensor units. Docs: RUNBOOK.md:195, README.md:194, README.md:195. Evidence: zig/src/native/cuda.zig:33, zig/src/families/gemma4/cuda_native.zig:21, CHANGELOG.md:881, zig/src/families/gemma4/model.zig:93, zig/src/families/gemma4/weights.zig:234, zig/src/families/gemma4/assistant.zig:95, zig/src/families/gemma4/dflash.zig:89, CHANGELOG.md:849. 11. Native --precision is family-specific: Super VL checkpoint math requires Blackwell and dense Qwen accepts full only. The two --prefill-fp8 flags remain Python-only. Docs: docs/recipes/cuda.md:8, docs/recipes/cuda.md:209. Evidence: zig/src/server/cli.zig:137, zig/src/server/cli.zig:138, zig/src/server/cli.zig:139, zig/src/server/cli.zig:303, zig/src/families/nemotron/cuda_latent_ck.zig:13, zig/src/families/nemotron/cuda_kernels.zig:190, zig/src/families/qwen3_5/cuda_native.zig:55. 12. The legacy checkpoint/precision/transport/serialization tables and Apple memory scopes are explicitly Python 0.6.6 history, not current native limits. Docs: docs/recipes/cuda.md:70, docs/recipes/cuda.md:167, docs/recipes/cuda.md:222, docs/recipes/cuda.md:329, docs/recipes/README.md:44. Evidence: CONTRIBUTING.md:4, zig/src/native/cuda.zig:131, CHANGELOG.md:795. 13. The segmented tensorfold run description applies to Nemotron; it is not a promise that every registered family takes four segments. Docs: docs/recipes/cuda.md:106. Evidence: zig/src/cli/cuda_main.zig:228, zig/src/families/qwen3_5/cuda_native.zig:31. 14. The P2P report describes native ranked implementation; its DeepSeek references do not imply the release serves that family. Docs: docs/recipes/cuda.md:356. Evidence: zig/src/native/cuda.zig:53, zig/src/families/glm/cuda_native.zig:230, zig/src/native/cuda_refusal.zig:5. 15. The Qwen3.6 MLX row records a CUDA run without describing native Metal as absent: Metal also registers qwen3_5_moe, while the linked recipe retains Python Metal commands. Docs: README.md:192. Evidence: zig/src/native/metal.zig:48, zig/src/native/metal.zig:179, CHANGELOG.md:1228, docs/recipes/qwen3.6-moe.md:3.
Correct the native support, rank and precision descriptions while keeping the frozen Python tables and measurements explicitly historical. Distinguish released images, kernel checks and checkpoint qualification. No implementation, test, lean baseline or measured speed changes. Claim audit against the main source: 1. Native CUDA registers more than Nemotron and Flash Next; registration is separate from device and checkpoint qualification. Docs: docs/recipes/cuda.md:11. Evidence: zig/src/native/cuda.zig:33, zig/src/native/cuda.zig:199, zig/src/native/cuda.zig:202. 2. Qwen4 is preparation with synthetic device gates and no real-checkpoint qualification; DeepSeek is refused pending gate #827. Docs: docs/recipes/cuda.md:13, RUNBOOK.md:201. Evidence: zig/src/native/cuda.zig:43, CHANGELOG.md:773, zig/src/native/cuda_refusal.zig:5. 3. Native GLM has recorded TP=2 and TP=3 serving, including FP4-Local and XSFP4; the CUDA setup is not limited to two ranks. Docs: docs/recipes/cuda.md:17, RUNBOOK.md:193, README.md:207. Evidence: zig/src/native/cuda.zig:33, zig/src/families/glm/cuda_native.zig:128, docs/receipts/glm-fp4local-80517fc3/README.md:27, CHANGELOG.md:517. 4. GLM uses TF_GLM_RANK, TF_GLM_WORLD and NCCL_COMM_ID; other families accept --tp only when their options consume ranks. Super VL has one- and two-GB10 receipts. Docs: docs/recipes/cuda.md:162, RUNBOOK.md:194. Evidence: zig/src/families/glm/cuda_native.zig:118, zig/src/families/glm/cuda_native.zig:180, zig/src/native/cuda_options.zig:12, zig/src/native/cuda_refusal.zig:55, docs/recipes/nemotron-3.5-super-vl.md:4. 5. EXL3 loading and DFlash2 drafting are native capabilities for particular families and formats, not Python-only features. Docs: docs/recipes/cuda.md:18. Evidence: zig/src/families/glm/cuda_exl3.zig:27, zig/src/families/glm/cuda_native.zig:291, zig/src/families/qwen3_5/cuda_native.zig:34, CHANGELOG.md:553. 6. The Nemotron MLX 4-bit serving record names GB10, RTX 3090 and RTX PRO 6000; an RTX 3090 receipt does not measure every sm_86 card. Docs: docs/recipes/cuda.md:20, docs/recipes/README.md:34, RUNBOOK.md:190. Evidence: zig/src/native/cuda.zig:33, zig/src/families/nemotron/cuda_native.zig:31, CHANGELOG.md:1662, CHANGELOG.md:1623, CHANGELOG.md:1538. 7. Release targets are sm_80, sm_86, sm_120 and sm_121. The A100 has recorded kernel checks but no served checkpoint on record, so it is not model-qualified. Docs: docs/recipes/cuda.md:22, docs/recipes/README.md:35, RUNBOOK.md:197, README.md:164. Evidence: zig/build/cuda.zig:542, CHANGELOG.md:1322. 8. Nemotron and dense Qwen have NVFP4/FP8 non-cluster fallback implementations for sm_80 to sm_89; no such model run on sm_80 is recorded. Docs: docs/recipes/cuda.md:24, docs/recipes/README.md:41, RUNBOOK.md:199, README.md:166, README.md:179. Evidence: zig/src/families/nemotron/cuda_kernels.zig:193, zig/src/families/qwen3_5/cuda_modelopt.zig:216, zig/src/cuda/qmmf.zig:153, CHANGELOG.md:1237, CHANGELOG.md:577. 9. Flash Next MLX 4-bit has a GB10 serving receipt; the ModelOpt/two-local-rank path is experimental, with tensor-rank drafting requiring the prompt cache off. There is no blanket all-SM qualification. Docs: docs/recipes/README.md:42. Evidence: zig/src/native/cuda.zig:33, zig/src/families/flashnext/cuda_lanes.zig:29, zig/src/native/cuda_refusal.zig:57, CHANGELOG.md:1221. 10. Gemma 4 12B and 31B MLX checkpoints, including assistant/DFlash drafting, have native CUDA GB10 receipts; pre-M5 Metal has a row path, while its assistant and DFlash drafters require tensor units. Docs: RUNBOOK.md:195, README.md:194, README.md:195. Evidence: zig/src/native/cuda.zig:33, zig/src/families/gemma4/cuda_native.zig:21, CHANGELOG.md:881, zig/src/families/gemma4/model.zig:93, zig/src/families/gemma4/weights.zig:234, zig/src/families/gemma4/assistant.zig:95, zig/src/families/gemma4/dflash.zig:89, CHANGELOG.md:849. 11. Native --precision is family-specific: Super VL checkpoint math requires Blackwell and dense Qwen accepts full only. The two --prefill-fp8 flags remain Python-only. Docs: docs/recipes/cuda.md:8, docs/recipes/cuda.md:209. Evidence: zig/src/server/cli.zig:137, zig/src/server/cli.zig:138, zig/src/server/cli.zig:139, zig/src/server/cli.zig:303, zig/src/families/nemotron/cuda_latent_ck.zig:13, zig/src/families/nemotron/cuda_kernels.zig:190, zig/src/families/qwen3_5/cuda_native.zig:55. 12. The legacy checkpoint/precision/transport/serialization tables and Apple memory scopes are explicitly Python 0.6.6 history, not current native limits. Docs: docs/recipes/cuda.md:70, docs/recipes/cuda.md:167, docs/recipes/cuda.md:222, docs/recipes/cuda.md:329, docs/recipes/README.md:44. Evidence: CONTRIBUTING.md:4, zig/src/native/cuda.zig:131, CHANGELOG.md:795. 13. The segmented tensorfold run description applies to Nemotron; it is not a promise that every registered family takes four segments. Docs: docs/recipes/cuda.md:106. Evidence: zig/src/cli/cuda_main.zig:228, zig/src/families/qwen3_5/cuda_native.zig:31. 14. The P2P report describes native ranked implementation; its DeepSeek references do not imply the release serves that family. Docs: docs/recipes/cuda.md:356. Evidence: zig/src/native/cuda.zig:53, zig/src/families/glm/cuda_native.zig:230, zig/src/native/cuda_refusal.zig:5. 15. The Qwen3.6 MLX row records a CUDA run without describing native Metal as absent: Metal also registers qwen3_5_moe, while the linked recipe retains Python Metal commands. Docs: README.md:192. Evidence: zig/src/native/metal.zig:48, zig/src/native/metal.zig:179, CHANGELOG.md:1228, docs/recipes/qwen3.6-moe.md:3.
Correct the native support, rank and precision descriptions while keeping the frozen Python tables and measurements explicitly historical. Distinguish released images, kernel checks and checkpoint qualification. No implementation, test, lean baseline or measured speed changes. Claim audit against the main source: 1. Native CUDA registers more than Nemotron and Flash Next; registration is separate from device and checkpoint qualification. Docs: docs/recipes/cuda.md:11. Evidence: zig/src/native/cuda.zig:33, zig/src/native/cuda.zig:199, zig/src/native/cuda.zig:202. 2. Qwen4 is preparation with synthetic device gates and no real-checkpoint qualification; DeepSeek is refused pending gate #827. Docs: docs/recipes/cuda.md:13, RUNBOOK.md:201. Evidence: zig/src/native/cuda.zig:43, CHANGELOG.md:773, zig/src/native/cuda_refusal.zig:5. 3. Native GLM has recorded TP=2 and TP=3 serving, including FP4-Local and XSFP4; the CUDA setup is not limited to two ranks. Docs: docs/recipes/cuda.md:17, RUNBOOK.md:193, README.md:207. Evidence: zig/src/native/cuda.zig:33, zig/src/families/glm/cuda_native.zig:128, docs/receipts/glm-fp4local-80517fc3/README.md:27, CHANGELOG.md:517. 4. GLM uses TF_GLM_RANK, TF_GLM_WORLD and NCCL_COMM_ID; other families accept --tp only when their options consume ranks. Super VL has one- and two-GB10 receipts. Docs: docs/recipes/cuda.md:162, RUNBOOK.md:194. Evidence: zig/src/families/glm/cuda_native.zig:118, zig/src/families/glm/cuda_native.zig:180, zig/src/native/cuda_options.zig:12, zig/src/native/cuda_refusal.zig:55, docs/recipes/nemotron-3.5-super-vl.md:4. 5. EXL3 loading and DFlash2 drafting are native capabilities for particular families and formats, not Python-only features. Docs: docs/recipes/cuda.md:18. Evidence: zig/src/families/glm/cuda_exl3.zig:27, zig/src/families/glm/cuda_native.zig:291, zig/src/families/qwen3_5/cuda_native.zig:34, CHANGELOG.md:553. 6. The Nemotron MLX 4-bit serving record names GB10, RTX 3090 and RTX PRO 6000; an RTX 3090 receipt does not measure every sm_86 card. Docs: docs/recipes/cuda.md:20, docs/recipes/README.md:34, RUNBOOK.md:190. Evidence: zig/src/native/cuda.zig:33, zig/src/families/nemotron/cuda_native.zig:31, CHANGELOG.md:1662, CHANGELOG.md:1623, CHANGELOG.md:1538. 7. Release targets are sm_80, sm_86, sm_120 and sm_121. The A100 has recorded kernel checks but no served checkpoint on record, so it is not model-qualified. Docs: docs/recipes/cuda.md:22, docs/recipes/README.md:35, RUNBOOK.md:197, README.md:164. Evidence: zig/build/cuda.zig:542, CHANGELOG.md:1322. 8. Nemotron and dense Qwen have NVFP4/FP8 non-cluster fallback implementations for sm_80 to sm_89; no such model run on sm_80 is recorded. Docs: docs/recipes/cuda.md:24, docs/recipes/README.md:41, RUNBOOK.md:199, README.md:166, README.md:179. Evidence: zig/src/families/nemotron/cuda_kernels.zig:193, zig/src/families/qwen3_5/cuda_modelopt.zig:216, zig/src/cuda/qmmf.zig:153, CHANGELOG.md:1237, CHANGELOG.md:577. 9. Flash Next MLX 4-bit has a GB10 serving receipt; the ModelOpt/two-local-rank path is experimental, with tensor-rank drafting requiring the prompt cache off. There is no blanket all-SM qualification. Docs: docs/recipes/README.md:42. Evidence: zig/src/native/cuda.zig:33, zig/src/families/flashnext/cuda_lanes.zig:29, zig/src/native/cuda_refusal.zig:57, CHANGELOG.md:1221. 10. Gemma 4 12B and 31B MLX checkpoints, including assistant/DFlash drafting, have native CUDA GB10 receipts; pre-M5 Metal has a row path, while its assistant and DFlash drafters require tensor units. Docs: RUNBOOK.md:195, README.md:194, README.md:195. Evidence: zig/src/native/cuda.zig:33, zig/src/families/gemma4/cuda_native.zig:21, CHANGELOG.md:881, zig/src/families/gemma4/model.zig:93, zig/src/families/gemma4/weights.zig:234, zig/src/families/gemma4/assistant.zig:95, zig/src/families/gemma4/dflash.zig:89, CHANGELOG.md:849. 11. Native --precision is family-specific: Super VL checkpoint math requires Blackwell and dense Qwen accepts full only. The two --prefill-fp8 flags remain Python-only. Docs: docs/recipes/cuda.md:8, docs/recipes/cuda.md:209. Evidence: zig/src/server/cli.zig:137, zig/src/server/cli.zig:138, zig/src/server/cli.zig:139, zig/src/server/cli.zig:303, zig/src/families/nemotron/cuda_latent_ck.zig:13, zig/src/families/nemotron/cuda_kernels.zig:190, zig/src/families/qwen3_5/cuda_native.zig:55. 12. The legacy checkpoint/precision/transport/serialization tables and Apple memory scopes are explicitly Python 0.6.6 history, not current native limits. Docs: docs/recipes/cuda.md:70, docs/recipes/cuda.md:167, docs/recipes/cuda.md:222, docs/recipes/cuda.md:329, docs/recipes/README.md:44. Evidence: CONTRIBUTING.md:4, zig/src/native/cuda.zig:131, CHANGELOG.md:795. 13. The segmented tensorfold run description applies to Nemotron; it is not a promise that every registered family takes four segments. Docs: docs/recipes/cuda.md:106. Evidence: zig/src/cli/cuda_main.zig:228, zig/src/families/qwen3_5/cuda_native.zig:31. 14. The P2P report describes native ranked implementation; its DeepSeek references do not imply the release serves that family. Docs: docs/recipes/cuda.md:356. Evidence: zig/src/native/cuda.zig:53, zig/src/families/glm/cuda_native.zig:230, zig/src/native/cuda_refusal.zig:5. 15. The Qwen3.6 MLX row records a CUDA run without describing native Metal as absent: Metal also registers qwen3_5_moe, while the linked recipe retains Python Metal commands. Docs: README.md:192. Evidence: zig/src/native/metal.zig:48, zig/src/native/metal.zig:179, CHANGELOG.md:1228, docs/recipes/qwen3.6-moe.md:3.
Piece (d) of the #690 split, rebased onto the 1.1.0 release (
76aafcf27) and stacked on the family and test pieces: the lane backend on the 1.1.0 lane core — one stream per slot, windows of rows — so prompt caching, concurrency and drafts come from the core. facts() reports exact_width = the plan's rows, drafts 0 (greedy only, as GLM-5.3-Flash landed), max_streams = lanes; drafts, grammar masks and tree requests return error.Unsupported until their pieces land.after.disk_read == before.disk_readover that full struct.--engram-table-dir, and the token map built at open when missing (the pull needs no Python step; the builder is the first piece).docs/deepseek-v41.md): what one Mac needs, the serve line, the lane semantics, the numerical picture.Rebase notes: the branch now rides 1.1.0's rewritten shared infrastructure —
paced_read's admission gauge replaces the olderwaitForRoomform, the tokenizer module is owned once by the engines registry, and the families table inmetal.zigis the merged one. A comment-only tidy commit brings the reference, family, native and Metal comments onto one line each within the 120-column rule.The usual checks, measured on the 2.8bpw pack served on one Mac: drafted == "draft": false (speculative.drafted 0 with drafts requested, content identical to --no-drafts), resumed == fresh (a conversation arriving across two calls equals the one-shot full-context reply), concurrent == solo (two lanes, Paris/Paris and 1024/1024). The lane receipt shows 0 bit disagreements between slots and their solo runs on both packs; each receipt carries
evidence_origin: current_invocation_not_historical_qualificationand a residency gate over the full rusage counters.Suites at this head:
test-deepseek-v4179/79 (41 steps),test-deepseek-v41-reference26/26,test-deepseek-v41-weights13/13,zig build native -Dcpu=apple_m154/54 steps,zig build testexit 0, 307/307 steps (4440/4455 cases passed, 15 pre-existing platform skips, fresh cache);tools/zig/lean_new.py501 problems against 503 on the 1.1.0 base, 0 new;zig fmt --checkclean; the built token map is byte-identical to the release's. The runner documentslanes MODEL TABLES IDS_JSON OUT_DIR [CHUNK] [CONTEXT]and a host test pins the accepted arities: six to eight arguments parse, five or nine are usage errors.Receipts live on the fork beside the branch; the tree carries the tests that regenerate them.
Status on the 1.1.2 line. This head is unchanged and still rebased onto
76aafcf27(96 commits behindd1018b318), so it shows as conflicting; it is not retargeted here. The shared lane host and token map already landed in 1.1.1; the DeepSeek-specific lane host, guide and mHC evidence in this PR stay the piece that waits on the serving gate tracked here. Refreshed family base for review:pr-dsv41-bc-family-11at4f69fea10. No action requested.