[ashhart]

Part of #1762: #1764's prompt-pass wins, ported to the other CUDA engines. #1764 (@Arminova) is on main; this reuses its tile rather than adding another.

• The BF16 prompt port, reconciled with flashnext cuda: prompt passes with the same bits and less work, and a prompt scratch sized by use #1764's landed tile. prompt16.cu and its records are untouched, tf_prompt_bf16/_f32 are defined once with one record each, and matmulBf16 is unchanged. What's added is a switch (TF_CUDA_PROMPT_BF16, read once) and a fixed 1,024-row gate, opted into by Qwen ModelOpt, Nemotron and Dory.
• Dory's projections on the BF16 tile: enableBf16() plus bf16_prompt = true, with a host ABI test (1,023 rows stay on the lane; 1,024 and 2,048 go to the tile; switch off and decode stay on the lane).
• Qwen prompt rows normalised on the GDN chain warps, and Qwen MoE router expert blocks scheduled first. Both are byte-identical to the kernels they replace, and registered in the kernel contract as replacements.

Exactness: on a GB10, the kernel contract for the GDN norm and router passed. Its only new FAIL lines are its four planted controls, and the limits are unchanged. Two independent reviews cover the code.

Device runs before landing (GB10):

• the kernel contract on this head;
• Dory: greedy and seeded token A/B against main, also with TF_CUDA_PROMPT_BF16=0;
• Qwen 3.5 27B NVFP4: token A/B from 2K to 64K.

Co-credit to @Arminova for #1764's tile and fusion.

[akol1]

While qualifying the #1136 BF16 prompt-scoring contribution on top of this head (b0b5beb), the full CUDA contract exited 139 immediately after cuda prompt16 FP8 tile 4, before a SUMMARY.

I found a callback-lifetime problem in contract_prompt_bf16.zig: Tile.kernel() returns .vt = &g.hooks(Tile), the address of a temporary returned table. A host regression requiring registration to retain static callback storage fails in both debug and optimized modes (46/48 tests passed before). Defining pub const callbacks = g.hooks(Tile) in Tile and returning &callbacks passes 48/48.

The fix and regression are in my local dependent contribution; the full GPU rerun is pending, so I am not yet calling this a complete crash-resolution receipt. No serving dispatcher, kernel, bound or precision changes are involved. I will retain the failed run and publish the final source-bound SUMMARY with the replacement contribution.

[ashhart]

Added @akol1's fix from #2020 as 789846adb, with their authorship: the contract registration took the address of a temporary g.hooks(Tile), so the callbacks dangled once registration returned. It now points at a callbacks constant, and a host test pins it. Thanks @akol1 for finding it.

[ashhart]

Device status. Dory and Qwen3.8-27B NVFP4 on a GB10 are GREEN, with tokens exact. The CUDA kernel contract on the earlier head (b0b5beb8d) died partway through with SIGBUS, while main's tree passed: 558 kernels, 0 fails, 362 controls red as planted. That fits the dangling callback pointer @akol1 found, whose fix is now 789846adb on this branch. The contract is re-running on 789846adb, and this waits on it.

[ashhart]

The CUDA kernel contract on 789846adb, with @akol1's callback fix, runs clean through the gates on a GB10: 568 kernels pass gates 1-4 and all 367 controls go red as planted. The earlier SIGBUS is gone. One thing is left before landing: launch limits show 214 problems against main's 190. Four launches have no records (gdn_prefill chain_kernel and tf_gdn_chain_norm, tf_moe_router_expert_first, tf_q35_gdn_post), and we're adding those.

[akol1]

Thanks — good to see the contract clean on 789846adb and the SIGBUS gone with the callback fix, and the GB10 Dory and NVFP4 results exact. The launch-limit delta (214 against main's 190) and the four launches without records (gdn_prefill chain_kernel, tf_gdn_chain_norm, tf_moe_router_expert_first, tf_q35_gdn_post) are clear; nothing needed from me, ping me when the records land if a second pair of eyes helps.

[ashhart]

Pushed 30e9b87, which fills the four launches that had no records ("recorded 0, 0") on the GB10 contract run. Two are new in this PR and two are older launches that this PR's collector and registry changes now expose. Their records for all five release SMs come from the nvcc 13.0.88 receipts. Records for later compilers stay pending rather than guessed, and kernel bytes and launch geometry are unchanged.

A host regression fails without the records and passes with them. All 112 host contract tests pass, and lean adds no new problems. The CUDA kernel contract is queued again on a GB10 on this head to confirm the limits count now matches main's.

[ashhart]

The GB10 kernel-contract rerun on 30e9b8728, against main at ef9fa700a:

	main	this PR
Kernels passing gates 1-4	558 of 558	568 of 568
Planted controls red	362 of 362	367 of 367
Limit problems	190	190
Registered launches, release SMs	1,482 (168 problems)	1,486 (168 problems)

The limits count now matches main's, and no limit line is new. The only new FAIL lines in the device contract are the five new controls going red as planted, plus the launch total. That was the last open item, so this goes in the landing queue.

[ashhart]

/atc exchange a4f391aba2ddcdda486be69e619c9ecbb133ffda48c15fa8df36665cbd7e5d9f

[ashhart]

Held from landing for now. On a GB10, Qwen3.8-27B MLX 4-bit, main against this head (runs ordered base, new, new, base): tokens are equal (0 of 20 prompt rows differ) and decode is level (1 and 4 streams within 0.1%), but prompts are a little slower almost everywhere:

• 2K: -0.1% to -0.6%; 8K: -0.0% to -0.2%; 16K: +0.0% to -0.4%;
• 32K: -0.3% to -0.5% (base 1,310.4 and 1,308.8 tok/s against 1,300.1 and 1,305.0);
• 64K: -0.2% in all four phases.

At 32K and 64K both new runs sit below both base runs. Nemotron's cells are mixed (±0.7%) with tokens equal. This PR's MLX-path changes are the two exact Qwen prompt kernels, so they're the first place to look. A Qwen prompt-speed PR should leave the 27B MLX path level, and we'll rerun once that's understood.

[ashhart]

Rebased onto main f0d8c8d50 and split, now 6743ec962. The GB10 hold on 6 Nov was the 27B MLX prompt running 0.1-0.6% slower, and the two exact Qwen prompt kernels from #1762 were the only change on that path. They now load only for ModelOpt checkpoints (qwen3_5/cuda_modelopt_prompt.zig). The MLX path keeps main's kernels and launches: all 272 existing CUDA kernel files are byte-identical to main's, and a host test pins the launch sequence against main's at 21 row and layer shapes. The BF16 prompt tile for ModelOpt, Nemotron and Dory is unchanged. The ten commits keep their authors, @akol1's callback-lifetime fix and Armin's co-credit included.

Next on our side: GB10 cells (27B MLX, expected level, and Nemotron for the tile) and the kernel contract for the new images. #2020 can rebase onto this once it lands.

• Synthena (Ash's Agent)

[ashhart]

Device check on GB10, main against c4920d4, on Dory with CELLS pp, sweep and chat:

• Tokens: every prompt-speed row's token hash equals main's, all 20 rows.
• Prompt speed: level. Every cell is within 0.4% of main's, from 2,048 to 65,536 tokens.
• Decode sweep: level.

An earlier RED on this run came from older files left in the same output folder. It wasn't this change.

The kernel contract failed only on 12 launch-limit records: the router and GDN chain kernels were recorded for sm_121 alone, so sm_80, 86, 89 and 120 held zero. a5aea9a records nvcc 13.0's register counts for all five chips, taken from the contract's own record lines. test-kernel-contract passes, 204/204. We'll re-run the contract on this head to confirm the limit count matches main's.

• Synthena (Ash's Agent)

[ashhart]

Kernel contract re-run on a5aea9a: launch-limit problems are 168, the same as main's, with no new-only limit failures. The one contract line that differs from main is the totals line, because this PR registers five more launches (1568 against 1563). Device A/B and contract are both level, so this is queued to land.

• Synthena (Ash's Agent)