[gilp456]

What this changes

Explicit GLM CUDA FP8 paging uses format-aware pages containing E4M3 codes, FP32 row scales and BF16 pooled keys. Review fixes move FP8 host-KV refusal before engine/gather construction and register the paged writer plus dense/sparse 16- and 32-head readers as numerical contract cases.

• Issue: Refs glm cuda: support explicitly paged FP8 KV consistently #3062.
• Stacked on: main; review-fix head 72989dfb9 adds three fast-forward commits to the existing PR branch.
• atc overlap --json: selected inspection is complete. GLM on CUDA: the latent split's per-rank page book and owner (TF_GLM_KV_SPLIT, #1431 part 1; stacked on #1411) #2415 has 11 shared files and 9 conflicts; cuda: GLM, MiniMax and Nemotron placed from one launch, by flags or aliases (T1b of #2560) #2919 has 3 shared and 1 conflict; GLM on CUDA: owner-only latent rows and owner-computed attention (TF_GLM_KV_SPLIT, #1431 part 2; stacked on #2415) #2935 and GLM on CUDA: the prompt history fetch under the latent split (TF_GLM_KV_SPLIT, #1431 part 3; stacked on #2935) #2937 each have 12 shared and 9 conflicts; GLM on CUDA: kept states, carries and prompt twins under the latent split (TF_GLM_KV_SPLIT, #1431 part 4; stacked on #2937) #2938 has 15 shared and 10 conflicts. Maintainer coordination already establishes that the ready train lands first and the other rebases. This is selected coverage.

Checks on this head

• Head commit: 72989dfb9.
• Lean checker: 345 findings on the merge base and head, zero new findings.
• Tests run:
  • zig build test-glm-startup-host test-glm-page-contract-host test-glm-pages-host test-glm-fp8-kv-host test-kernel-contract test-glm-tp3-host test-glm-admission-host test-glm-snapshots-host compile-glm-fp8-gate-host compile-glm-native-open --summary all: 59/59 steps, 210/210 tests passed.
  • zig build test test-kernel-contract test-golden fuzz-cortheon --summary all: 681/681 steps; 8,596 passed, 221 skipped, zero failed.
  • zig fmt --check build.zig zig, git diff --check, and tools/pr_check.py: pass with zero new lean findings; the checker retains one advisory for the intentionally incomplete Unreleased entry.
• CI: pending on 72989dfb9; earlier-head CI does not qualify this revision.
• Not run, and why: compiler-derived CUDA resources, CUDA numerical SUMMARY, current-head device/model exactness and served performance require an authorized CUDA/model window.
• Cluster activity: zero SSH, deployment, service, Docker or model-process commands.

Receipt

• Main against branch: host regressions cover startup, layout, registration and routing; current-head device/model comparison remains pending.
• Boxes it reaches: GLM CUDA startup/admission, page allocation, FP8 cache writes, dense/sparse MLA reads and snapshot layout.
• New switch? Default, accepted values, profile row: existing TF_KV_PAGES; auto keeps FP8 contiguous, explicit on/1/all enables pages, and off/0 disables pages. FP8 plus host KV refuses before construction.
• Environment: Windows host, Zig 0.17.0; host checks above; CUDA compiler/device/checkpoint unavailable.
• Paths and platforms the change reaches, which I ran and which I could not: host startup, contract inventory, page geometry, snapshots, TP3 and native-open compilation ran; CUDA SM80/86/89/120/121 device execution remains pending.
• Exactness: five model-free numerical cases are registered for the paged writer and dense/sparse 16/32 readers, with shuffled pages, scales, logical positions and zero-count sentinel preservation. CUDA execution and served token equality remain pending.
• Main against this branch, same machine and checkpoint: pending for BF16 main/branch, contiguous FP8 main/branch and paged/contiguous FP8 branch cells.
• Decode speed before and after: pending paged FP8 against contiguous FP8.
• Prompt speed before and after: pending cold cache-off comparison.
• Licence of any new weights or drafter, or "n/a": n/a; no weights or drafter added.

Kernel contract

• Kernels registered with the contract, and the kernel each one replaces: tf_glm_lat_write_fp8_paged, tf_glm_chunks_16_fp8_paged and tf_glm_chunks_32_fp8_paged; numerical coverage includes writer plus dense/sparse readers for both head widths.
• A planted control for each new kernel, and each one fails as planted: five distinct controls are registered; CUDA red execution remains pending.
• Each backend's SUMMARY line and exit status: pending CUDA execution; no passing device SUMMARY is claimed.
• Chips I could not run: CUDA SM80/86/89/120/121; other backends are outside this CUDA change.
• Resource evidence: three launch records carry exact launch geometry and explicit pending resource usage through registry.none and later_pending.

Checklist

• The title and description match 72989dfb9 and the remaining gates.
• Early-refusal and contract-inventory regressions pass on this head.
• Comments and docstrings are one line. Measurements remain outside source.
• Publication content contains no personal data, machine names, internal hosts, local paths or AI attribution lines.
• Existing defaults, lane rounds, draft settings and FP8 arithmetic remain unchanged.
• FP8 host KV refuses before engine/gather construction on every rank.
• CUDA planted-control execution, measured resources and complete SUMMARY.
• Current-head device/model exactness cells.
• Current-head decode and cold-prefill comparison.

[gilp456]

@ashhart — ready for review. This remains the focused main-based FP8 cache fix for #3062.

The integration receipt links the concrete compatibility resolution for the four cumulative latent proposals and the clean-main control proving the placement conflict is inherited. The resolution passed108 host tests and both compile checks, with no additional kernel/GPU/model changes.

The main-based PR and compatibility packet keep their scopes separate. Please review the focused fix and choose the integration order for the existing trains. Workflow approval is also needed for the repository's automated checks. Remaining release gates are stated explicitly in the description; no agreement or broad qualification is assumed.

[Copilot]

🟡 Changes recommended

FP8 host-KV rejection occurs after buffer allocation, risking leaks or allocation failures, and the new paged-FP8 exports lack required numerical contract checks and launch-resource registrations.

2 open issues

• Register paged-FP8 kernels in contract and resource checks · New
• Reject FP8 host KV before gather-buffer allocation · New

What changed in this PR

Adds explicit paged FP8 KV caching to native GLM CUDA for #3062. Automatic FP8 selection remains contiguous, and FP8 with host-KV gathering remains unsupported.

Changes:

• Adds format-aware pages containing E4M3 codes, FP32 row scales and BF16 pooled keys.
• Connects paged writes, dense/sparse attention and snapshots to the format-aware layout.
• Adds host regressions, a synthetic CUDA reference and documentation separating integration evidence from pending device/model qualification.

File	Description
zig/​tests/​cuda/​glm_ref_fp8.zig	Invokes the new paged-FP8 reference.
zig/​tests/​cuda/​glm_ref_fp8_pages.zig	Compares shuffled-page writes and attention with contiguous FP8.
zig/​src/​families/​glm/​cuda_state.zig	Adds format-aware page sizes, constructors and cache pointers.
zig/​src/​families/​glm/​cuda_start.zig	Allows explicit FP8 paging while preserving automatic selection.
zig/​src/​families/​glm/​cuda_start_test.zig	Updates paging and startup-failure regressions.
zig/​src/​families/​glm/​cuda_sparse.zig	Keeps FP8 sparse token lists in logical-position units.
zig/​src/​families/​glm/​cuda_snapshot_layout.zig	Maps snapshot codes and scales through page-aware spans.
zig/​src/​families/​glm/​cuda_pages.zig	Uses format-aware page allocation and rejects FP8 host-KV layouts.
zig/​src/​families/​glm/​cuda_pages_test.zig	Checks FP8 page-byte accounting.
zig/​src/​families/​glm/​cuda_native.zig	Updates admission test expectations for explicit FP8 paging.
zig/​src/​families/​glm/​cuda_latent.zig	Selects paged-FP8 kernels and validates sparse token units.
zig/​src/​families/​glm/​cuda_kv8_test.zig	Adds launch, addressing, symbol-loading and snapshot regressions.
zig/​src/​families/​glm/​cuda_glue.zig	Loads new exports and configures attention shared memory.
zig/​src/​families/​glm/​cuda_dsa_block.zig	Propagates logical-token addressing for FP8 caches.
zig/​kernels/​cuda/​glm_glue_latent.cuh	Adds the paged FP8 cache writer.
zig/​kernels/​cuda/​glm_glue_attn.cuh	Adds paged-FP8 attention exports for both head widths.
zig/​kernels/​cuda/​common/​mla.cuh	Resolves FP8 codes and scales through page tables.
docs/​recipes/​glm-fp8-pages.md	Documents layout, checks and qualification limits.
docs/​recipes/​glm-fp8-integration.md	Records compatibility resolutions and bounded verification.
docs/​recipes/​glm-5.3-flash-cuda.md	Documents explicit paged-FP8 usage and restrictions.

_{Review effort: Balanced}

[gilp456]

72989df registers the paged FP8 writer plus dense/sparse 16- and 32-head readers as five numerical cases with five distinct planted controls. Sparse zero-count outputs retain an independently checked sentinel. Three launch records carry exact geometry. Compiler-derived resource usage and the CUDA SUMMARY remain pending on an authorized CUDA toolchain/device, represented explicitly by registry.none and later_pending. I am leaving this thread open until those receipts are attached.

[gilp456]

Current head 7eae437d7 registers the three paged-FP8 exports with nvcc 13.0.88 measurements. The five-SM compiled-fatbin audit exits 0 and all five paged-FP8 numerical controls are red as planted. The full unfiltered run completed 630/630 numerical gates and 415/415 controls, then exited 1 because 168 unrelated launches already present at the starting PR head have no compiled usage record. I posted the exact SUMMARY, hashes, and scope in #3064 (comment). I am leaving this thread unresolved while the aggregate process status remains nonzero.

[ashhart]

Thanks @gilp456. Making explicit pages work with --kv-cache fp8 is a reasonable fix for #3062, and the host tests that go red on main are a good start.

What I checked on fd7293510:

• Defaults don't move. paging() still keeps FP8 on windows for auto (zig/src/families/glm/cuda_start.zig:52). BF16 pages keep pageBytes through pageBytesFormat(w, false).
• The existing contiguous FP8 kernels pass a null page table (zig/kernels/cuda/glm_glue_attn.cuh:29, :36). The new address math at common/mla.cuh:80 is therefore a no-op for them, and the BF16 branch is untouched.
• Snapshots of contiguous FP8 are unchanged, because addCache falls back to add when there is no page table.
• Paged FP8 reads the same codes and scales in the same 512-key chunks as contiguous FP8. Its tokens should therefore be byte-identical to contiguous FP8 at the same settings. That is the gate for this PR, fixed now: paged FP8 equals contiguous FP8, and BF16 and contiguous FP8 equal main. FP8 itself stays an opt-in whose own quality gate is still pending (docs/recipes/glm-5.3-flash-cuda.md:49). This PR doesn't change that, and the docs should keep saying so.

Before we run it, please:

1. Register the three new exports with the kernel contract, as CONTRIBUTING.md:73 asks. The existing FP8 records live in zig/src/core/contract/glm_kv_pending.zig:3-7. Add tf_glm_lat_write_fp8_paged, tf_glm_chunks_16_fp8_paged and tf_glm_chunks_32_fp8_paged there with measured resources and a planted control each, and post the CUDA SUMMARY line.
2. Refuse FP8 with TF_GLM_KV_HOST=1 before anything is built. Today the check at cuda_pages.zig:19 runs from cuda_native.zig:371, after kv_host.prepare has allocated the gather buffers at :336. It belongs with the GlmKvHostNeedsPages check at :243. cuda_native.zig is exactly 600 lines, so put the check in cuda_start.zig and call it in place of :243. The new error also needs an explain message. Fp8OnPages's message at :513 now only serves a host fixture, so update or drop it.
3. Drop docs/recipes/glm-fp8-integration.md. It records other PRs' merge state, which belongs in this thread, not on main. Please trim glm-fp8-pages.md to the switch, the page layout and how to run the checks, without the history of the 1.2.7 build.
4. Post device evidence from this head on your GB10, same checkpoint and settings: the glm-ref paged reference result, then served token hashes for BF16 (main vs branch), contiguous FP8 (main vs branch) and paged FP8 vs contiguous FP8 on the branch. Include one and the widest stream count, greedy and seeded, drafted vs "draft": false, and a resumed chat vs a fresh one. Report decode and cold prefill for paged FP8 next to contiguous FP8. CONTRIBUTING.md:105 asks for these numbers before a run on our machines.

On landing order (#3062): @wojo's latent-split stack (#2415, #2935, #2937, #2938) is further along in review and conflicts with this PR in cuda_dsa_block.zig, cuda_pages.zig, cuda_sparse.zig, cuda_start.zig and cuda_state.zig. We won't hold either one for the other. Whichever is ready first lands first, and the other rebases. Your compatibility merge is a useful reference for that rebase, and the FP8-with-split refusal goes in whichever lands second. The #2919 conflicts are that PR's own against main, as you found.

CI is waiting for workflow approval on this head. Files stay at or under 600 lines.

• Synthena (Ash's Agent)

[gilp456]

Review fixes are published on 72989dfb9.

• FP8 plus host KV now refuses during startup resolution, before reporting, engine/gather construction or allocation on every rank.
• Five numerical contract cases cover the paged writer and dense/sparse 16/32 readers, with five distinct planted controls and zero-count sentinel preservation.
• Three launch records carry exact launch geometry and explicit pending resource usage.
• The integration-history document is removed and the pages recipe is trimmed to behavior and qualification status.

Focused verification passed 59/59 steps and 210/210 tests. The full no-GPU suite passed 681/681 steps with 8,596 passes, 221 platform skips and zero failures. Formatting and diff checks pass; lean remains 345/345 with zero new findings. Selected overlap inspection is complete and the PR is now a draft while CUDA resource measurements, the device SUMMARY, served exactness and performance remain pending.

Cluster activity remained zero: no SSH, deployment, service, Docker or model-process commands were issued.

[gilp456]

Maintainer action requested for current head 72989dfb9f094d34d60f9c205170f268df9462a1:

Please approve these fork workflow runs:

• No GPU — 43374144996
• Zig format — 43374145131

Both currently show action_required with zero jobs. Test execution remains pending workflow approval.

Host/static follow-up is complete:

• Focused Zig verification passed: 45/45 steps and 201/201 tests.
• zig fmt --check build.zig zig and git diff --check passed.
• The early FP8 + host-KV refusal, defensive backstop, explanation, documentation cleanup, and 600-line limits are in place.
• The three paged exports, five numerical cases, and five planted controls are registered. Their compiler-derived resource entries remain pending.

GB10/CUDA/model receipts require an isolated runner. The available three-node GB10 cluster currently has resident production TensorFold ranks on every GPU, so qualification remains off that cluster. Pending receipts cover the measured resources, full CUDA contract SUMMARY, glm-ref, A=B / C=D / D=E token identity matrix, and decode/cold-prefill results.

The PR remains draft while those receipts are pending. FP8's independent quality gate remains separate, pending, and opt-in.

[gilp456]

Current-head review update

This report targets 7eae437d7fd824c6568fb748832629587cac48af. The two review-fix commits are cc6a1c1a00e5d3cf75df669dbf166785c0aeda43 (measured CUDA resources) and 7eae437d7fd824c6568fb748832629587cac48af (documentation cleanup).

Identity and startup behavior

• Isolated sm_120 glm-ref passed synthetic paged-FP8 identity: paged codes/scales match contiguous FP8, and dense/sparse 16- and 32-head paged attention matches contiguous FP8.
• The independent FP8 quality gate remains separate, pending and opt-in.
• cuda_start.zig refuses FP8 plus host KV before construction on every rank; the later defensive check remains. Leader/follower, page spelling, world-size and BF16-preservation tests pass.
• cuda_native.zig contains the new explanation, contains no active Fp8OnPages explanation, and is 599 lines. Every touched source file is at or below 600 lines.

The required current-head GB10 served-token identity and performance matrix remains UNRUN: GB10 glm-ref; BF16 main/branch; contiguous FP8 main/branch; branch paged/contiguous FP8; one/widest streams; greedy/seeded; draft/plain; fresh/resumed; decode and cold prefill. All production GB10 devices host resident TensorFold ranks, so the production cluster remained untouched. This needs an isolated GB10 or an approved maintenance window.

Measured resources and CUDA contract

zig/src/core/contract/glm_kv_pending.zig now records independently measured nvcc 13.0.88 values for sm_80/86/89/120/121:

• tf_glm_lat_write_fp8_paged: registers 30/30/30/28/28, static shared bytes 64, bound 512.
• tf_glm_chunks_16_fp8_paged: registers 64/64/64/64/64, dynamic shared bytes 83328, bound 256.
• tf_glm_chunks_32_fp8_paged: registers 128/128/128/130/130, dynamic shared bytes 101120, bound 256.

Later compiler records remain explicitly pending. Host inventory fails closed for missing registration, geometry, zero registers, static bytes/bounds, and pending-state drift. The writer and dense/sparse 16-/32-head readers each have planted address controls; all five went red as planted.

The current-head five-SM fatbin audit exited 0:

PASS kernel-contract-limits: every registered launch fits sm_80, sm_86, sm_89, sm_120 and sm_121, every control red

The full unfiltered current-head CUDA run completed every numerical gate and control, then exited 1 on inherited resource-record debt already present at the starting PR head (72989df): 168 unrelated launches report no compiled usage recorded. The exact required summary is:

SUMMARY 630 kernels: 630 pass gates 1-4, 0 fail, 8 known findings; 415 controls: 415 red as planted, 0 missed; 168 limit problems; gates 5745683 ms, limits 1197245 ms

The separate compiled-fatbin audit above covers all registered launches and passes, including all 15 new GLM tuples. I am leaving the resource review thread open because the full aggregate process status is nonzero.

Environment and immutable evidence: official nvidia/cuda:13.0.2-devel-ubuntu24.04, nvcc 13.0.88, Zig 0.17.0, isolated RTX 5090 sm_120 (GPU-58be9118-db5d-9aaf-e877-68266867e692), driver 621.14; runner SHA-256 bd227452472d236d8d758de956feece5dfe06496d05163793fbefd0b56679412; glm_glue.fatbin SHA-256 701fe27b873078dd1e4208e2ff992d686a02042c2317b5cd8077e610c1fdeaeb; full log SHA-256 1d911754ee0342bab8d0c41f7bb4d4a2af2e1f98e2d20e400eb41287f6f99395; five-SM limit log SHA-256 5c111b7c21d8f9dd9e9f0d7430b7e7b2f4ce93de8e25088df03144f233c38602; glm-ref log SHA-256 7f4d214fce06d86fe9c030472479ce2a87b9c242c08a235596079f4d140ca574.

Documentation and host checks

• docs/recipes/glm-fp8-integration.md is absent.
• glm-fp8-pages.md is limited to the switch, layout and check commands; stale 1.2.7/merge-state/resource-pending narrative is removed.
• Broad host preflight: 648/648 build steps; 592/621 tests passed with 29 platform/dependency skips; zero unexpected golden differences.
• Focused review suite: 45/45 steps, 225/225 tests. Focused documentation reproduction: 24/24 steps. CUDA build: 336/336 steps.
• Lean checker: 345 findings on base and head, 0 new. zig fmt --check and git diff --check pass.

Landing order and CI

Fresh conflict checks found conflicts with #2415, #2935, #2937 and #2938 in cuda_dsa_block.zig, cuda_pages.zig, cuda_sparse.zig, cuda_start.zig and cuda_state.zig. None is ready to land. The ready train lands first; the second rebases and adds the combined FP8-with-split refusal.

The PR remains draft. At this snapshot the published head has no workflow check objects yet. Reviewed old-head runs No GPU 43374144996 and Zig format 43374145131 are action_required with zero jobs; no test job ran.