[javierbits]

What this changes

A non-final Qwen 3.5/3.6/3.8 CUDA prompt chunk whose rows nothing reads no longer runs its last layer's query projection, attention, gate, output projection, MLP and head. Keep gains none, Shape gains unread, and cuda_prefill.zig passes none when no last-layer tap, learning/MLP/pocket reader, prompt int8 path or image chunk reads the chunk. The final chunk is unchanged.

Review follow-up on b2efa807e; rebased onto current main after #2838 landed, and the device receipt was rerun on the rebased binaries:

• Added a host launch trace. A test now drives cuda_forward.body over a fake engine and asserts an unread last layer launches norm, K/V, two head norms and RoPE, then stops. It goes red when the return moves above RoPE (missing rope) or is disabled (extra attention, o, norms and head).

• Filled the device plan: seeded sampled replies, the default drafter path, and resumed chats at the 8K and 16K prompt-cache marks. All are byte-equal between main and this branch.

• Dropped the unused Shape.key bit and its test; prompt shapes never reach the graph cache.

• Receipt now names Python 3.12.3 as the prompt source and says the unread=true log line came from a temporary std.log.info, not the diff.

• pr_check.py now runs with --pr 3060; the changelog bullet names qwen3_5 cuda: a prompt chunk nothing reads stops its last layer at its keys #3060.

• Point 6 is in: @jschmied ran the quiet-GB10 base-new-new-base A/B. At 32K the median per-round delta is -28.4 ms and the median adjacent pair is -34.5 ms (-0.5..-0.6 %), 0 of 6 adjacent pairs slower; 8K gains about 3-4 ms and 2K is level, with equal tokens. The ledger rows now carry those numbers.

• Issue: Fixes Qwen 3.5 CUDA: a prompt chunk nothing reads stops its last layer at the keys #2941.

• Stacked on: main.

• atc overlap --json: no open-PR overlaps. Three local, unpushed worktrees (port/1806-gdn-silu-gate, decode-share, nvfp4-refusal) report conflicts in CHANGELOG.md, cuda_prefill.zig and cuda_engine.zig; all predate current main. ATC writes return incomplete because the repository's 321 open PRs exceed ATC 0.9.6's 300-PR page cap, so this PR with Fixes #2941 is the claim.

Checks on this head

• Head commit: e34a57994.
• Lean checker on base and head: 345 both, 0 new findings. python3 tools/pr_check.py --base origin/main --pr 3060: 0 errors, 0 warnings.
• Tests run:
  • zig build test-qwen35-prefill-host --summary all: 19 passed, 0 failed, 0 skipped (includes the new launch trace).
  • zig build test-cuda-host --summary all: 391/391 steps, 1082 passed, 3 skipped, 0 failed.
  • zig fmt --check build.zig zig and git diff --check: clean.
  • Planted failures for the launch trace: moving the return above RoPE fails on the missing rope; disabling the return fails on the extra post-attention launches.
• CI: pending on this head after the latest main rebase.
• Not run: full zig build test test-golden (macOS-only checks unavailable here); no GPU kernel changed, so no kernel-contract run.

Receipt

• Main against branch: cold 2K/8K/32K, seeded sampled, default-drafted and resumed 8K/16K chats all have equal token and output hashes. The GB10 also served an unrelated 86 GiB model, so cold TTFT is reported but is not a clean performance gate; @jschmied is running the quiet-GPU A/B.
• Boxes it reaches: Qwen 3.5/3.6/3.8 CUDA prompt prefill, whole and stepped, with and without prompt cuts. The skipped path is the same engine forward.
• New switch: none.
• Environment: TensorFold base ab78aef90 (device runs on 0591c4e85, head 4a7a7f127), head e34a57994; Zig 0.17.0; CUDA 13.0 (V13.0.88); NVIDIA GB10 sm_121; checkpoint nvidia/Qwen3.6-35B-A3B-NVFP4 revision 491c2f1ea524c639598bf8fa787a93fed5a6fbce; --parallel 1 --context 40960. Binary md5 of the measured pair: base 6720d18b…, head fe92ed6a….
• Paths and platforms: ran CUDA sm_121 on GB10; did not run Metal, HIP, other CUDA SMs or Windows.
• Exactness: cold token_sha256/output_sha256 equal at 2,048, 8,192 and 32,768 tokens; seeded T 1.0 seed 1234 and default-drafted rows equal at 8,192 and 32,768; resumed 8K and 16K chat replies equal (e1de42ac… / 19079cfe…) to each other and to a cache-off run. Raw runs are in docs/receipts/qwen-unread-chunks-2941/.
• Main against this branch: cold TTFT base 0.340 / 1.284 / 6.045 s, head 0.347 / 1.272 / 5.953 s at 2K / 8K / 32K, with a clean base re-run after builds; the difference is inside shared-GPU noise.
• Decode speed: n/a, unchanged path; cold replies above used greedy 16-token replies.
• Prompt speed: quiet GB10, base a3aecd8d0 against head e7c385e96 (same zig/ diff as this head): 32K base 5.8006 s to head 5.7695 s, median per-round -28.4 ms and adjacent -34.5 ms (-0.5..-0.6 %), 0 of 6 adjacent pairs slower; 8K -3..-4 ms; 2K level; tokens equal. Reported by @jschmied.
• Licence of any new weights or drafter: n/a.

Checklist

• The title and description match the head commit.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Every token still goes through the lane rounds. Nothing needs drafts off.
• Drafted output equals plain, a resumed prompt equals a fresh one, and cold replies equal main.
• No precision traded for speed; the final chunk is byte-identical and the skipped rows are unread.
• Decode unchanged; prompt processing is level within noise.

[ashhart]

Thanks @javierbits. The change does what #2941 planned, and the code reads correctly to me.

What I checked on b2efa807e:

• The early return at zig/src/families/qwen3_5/cuda_forward.zig:87 comes after the K and V projection, both head norms and RoPE (:79-85), so the last layer still writes its keys and values.
• land copies nothing for .none and still calls recurrent.completed (cuda_engine.zig:533, :542).
• unread() (cuda_prompt_tail.zig:36) is false whenever something taps the last layer. The MTP drafter does (cuda_mtp.zig:121), so with drafts on nothing changes.
• The final chunk still passes .last (cuda_prefill.zig:159-160). Prompt-cache saves at marks copy state and KV, not the logits row, so a non-final chunk leaving that row stale is safe.

Before we run it, please:

1. Update the description. It names head 5fd142a6c, but the PR head is b2efa807e. Please confirm the receipt binaries came from b2efa807e. The pr_check.py line should say --pr 3060, not 2941.
2. Add the launch-trace test Qwen 3.5 CUDA: a prompt chunk nothing reads stops its last layer at the keys #2941 asked for. An unread chunk's last layer should launch K/V, the two head norms and RoPE, and nothing after them. The new tests check when .none is chosen, but not what body launches, so a return in the wrong place would still pass. The test should go red if the return moves above RoPE or below the attention.
3. Fill the rest of Qwen 3.5 CUDA: a prompt chunk nothing reads stops its last layer at the keys #2941's device plan: seeded (sampled) replies beside the greedy ones, state and KV bytes at the 8K and 16K prompt-cache marks equal to main's, and one cold row with the default drafter on.
4. Say where the unread=true log line in the receipt README came from. The diff adds no such line. A local debug print is fine, just say so.
5. Pin the prompts. gate.py:15-16 and :47-48 read the local Python standard library, so the tokens depend on the Python version. Name the version in the README, or use the public prompt set (CONTRIBUTING.md:83). Also, the error at gate.py:54 says "cold or resume" but the mode is called chat.
6. The GPU was shared, so the TTFT rows can't show the 35-60 ms at 32K the issue expects. A base-new-new-base cold A/B at 8K and 32K on a quiet GB10 is still needed. @jschmied asked about this issue earlier the same afternoon and offered to run that side. If you both agree, his A/B could complete this receipt. Once there is a measured gain, the two ledger rows (docs/optimization-ledger.md:269-270) should state it.

A small optional one: prompt shapes never reach the graph cache (cuda_shapes.zig:103), so the key bit at :30 is never read. Bit 24 is also inside the slot field that starts at bit 19, and it is free only because slots stop at 16. Dropping the bit and its test would be simpler.

Overlaps: this conflicts with #2838 in cuda_prefill_test.zig, and with #2620 and #2939 on the same key() line (they use bit 26, so the bits don't collide). Whichever lands second rebases. #2941 builds on #2838 and #2873. Basing this on main is fine, but please rebase when they land.

The default path keeps its tokens on everything I can read: the final chunk is unchanged and every skipped row is one nothing reads. Files stay under 600 lines. CI is green on this head.

• Synthena (Ash's Agent)

[jschmied]

@javierbits @ashhart I can run point 6, the cold A/B, on a quiet GB10 if that suits you, javierbits.

• Arms: main 6c5cef894 (your base) against your head, base-new-new-base, 3 rounds, cold 8K and 32K prompts. Nothing else on the GPU, clock capped at 2300 MHz, page cache dropped before every server start.
• Same setup as your receipt: nvidia/Qwen3.6-35B-A3B-NVFP4 at 491c2f1ea524c639598bf8fa787a93fed5a6fbce, --parallel 1 --context 40960 --no-drafts, greedy 16-token replies, md5 of each served binary, raw runs posted.
• Head: b2efa807e as it stands, or tell me the head to use if you rebase or add the launch-trace test first.

I'd post TTFT median with [min-max] per arm and the token hashes. Your other points (description, test, seeded rows, KV bytes, prompts) stay yours.

[javierbits]

@ashhart thanks for the detailed read. All code and receipt points are addressed on head 654f6a992:

1. Description and receipt now name 654f6a992; the binary in the receipt is that head (md5 3c856a9477ba87f8c12005206a924604). pr_check.py runs with --pr 3060, and the changelog bullet names qwen3_5 cuda: a prompt chunk nothing reads stops its last layer at its keys #3060.
2. Added the launch trace. cuda_launch_trace.zig records tags instead of issuing kernels, and cuda_prefill_test.zig drives cuda_forward.body over a fake engine. It asserts the unread last layer launches norm, K/V, two head norms and RoPE, then stops. Planting the return above RoPE fails on the missing rope; disabling it fails on the extra attention, o, norm and head launches.
3. Device plan filled on the same checkpoint: seeded T 1.0/seed 1234 and default-drafted rows at 8K/32K equal to main; resumed chats at the 8K mark (cached 8,192) and 16K mark (cached 16,384) equal to main and to a cache-off run.
4. The unread=true line is documented as coming from a temporary std.log.info in cuda_prefill.chunk for the run; the diff has no log line.
5. Prompts are pinned in the receipt to Python 3.12.3's standard-library source; the gate.py error text now says cold, sampled, drafted or chat.
6. @jschmied yes, please take the quiet-GB10 base-new-new-base A/B. Use 654f6a992 (binary above) against base 6c5cef894. My one clean pair here is 32K 6.045 s base to 5.967 s head on a shared GPU, so I have not changed the ledger gain; your run can set it.

Optional bit: dropped Shape.unread from key() and removed its test; the field stays for the body's decision.

[javierbits]

Rebased onto current main 0591c4e85 after #2838 landed; head is now 4a7a7f127 (binary md5 fe92ed6a93d7ede5c7a6bb6fa42a3cff). The #2838 conflict in cuda_prefill_test.zig is resolved by keeping its whole/readout fixtures and adding the unread reader fields and launch trace on top.

The receipt was rerun on the rebased binaries: cold 2K/8K/32K, sampled, drafted, and the 8K/16K cache marks all still equal base. @jschmied please use 4a7a7f127 for the quiet A/B rather than 654f6a992.

Checks on this head: test-qwen35-prefill-host 21/21, test-cuda-host 393/393 steps (30 skipped), lean 341 vs 341 with 0 new. CI is running.

[javierbits]

Final head after the latest main rebase: 73f35a6b0 (mergeable; CI running). The qwen3_5 sources are unchanged from the measured 4a7a7f127; the rebase only brought in the slide-rollback commits. @jschmied please use 73f35a6b0 for the quiet A/B.

[jschmied]

@javierbits @ashhart Here is the quiet-GB10 cold A/B (point 6). The head saves about 30 ms at 32K, 0 of 6 adjacent pairs slower, and the tokens are equal everywhere.

Prompt	base TTFT median [min-max]	head TTFT median [min-max]	paired head - base, per round	adjacent pairs slower
2,048	0.3275 s [0.3238-0.3291]	0.3259 s [0.3224-0.3272]	+1.8 / -4.5 / -2.7 ms	3 of 6
8,192	1.2360 s [1.2353-1.2437]	1.2328 s [1.2273-1.2356]	-3.1 / -9.9 / -1.5 ms	0 of 6
32,768	5.8006 s [5.7908-5.8210]	5.7695 s [5.7437-5.7825]	-28.4 / -61.8 / -23.0 ms	0 of 6

• At 32K, the median of the per-round deltas is -28.4 ms and the median of the six adjacent pairs is -34.5 ms (-0.5 to -0.6 %). That is at the low end of the 35-60 ms Qwen 3.5 CUDA: a prompt chunk nothing reads stops its last layer at the keys #2941 expected. 8K gains about 3-4 ms (one unread chunk). 2K is level, as expected with no unread chunk.
• token_sha256/output_sha256 are equal across all 12 servers at every length, and match your receipt: 0551b205…/4dfc61c7…, eabb9ee4…/bfda9f68…, bfdffc1b…/3e1bd53e….

Setup:

• Arms: base a3aecd8d0 against head e7c385e96, which was the PR head when I started.
• Builds: zig build native -Dsm=121 -Dnvcc=/usr/local/cuda-13.0/bin/nvcc (CUDA 13.0 V13.0.88, Zig 0.17.0). Binary md5: base 1375faa01aeefc2f9ac40c907d6dab81, head a479e68ff08992d57ff66f15faf52e3f.
• Head ce33fd6a7 was pushed after these runs. Its zig/ diff against its base ab78aef90 is the same as the measured one, apart from hunk offsets. Main's commits from a3aecd8d0 to ab78aef90 touch nothing under zig/src/families/qwen3_5 or zig/src/cuda.
• Order: base, head, head, base, 3 rounds, one fresh server per arm. Nothing else was on the GPU, the SM clock was capped at 2300 MHz (2281-2288 MHz read before and after each arm), and the page cache was dropped before every server start.
• Model: nvidia/Qwen3.6-35B-A3B-NVFP4 at 491c2f1ea524c639598bf8fa787a93fed5a6fbce.
• Server flags: --parallel 1 --context 40960 --no-drafts --prompt-cache-gib 0 --no-update-check. The load line says "prompts in passes of up to 4096 rows, no prompt cache" and "drafts: off".
• Prompts: your gate.py cold from the head, 16 greedy tokens, on Python 3.12.3, the version your README pins.

The raw rows (per-arm JSON, gate and server logs, runner) are on our side and I can attach them. With a measured gain, the ledger rows at docs/optimization-ledger.md:269-270 could say "-28 to -35 ms (-0.5 %) cold at 32K, -3 ms at 8K on GB10".

[jschmied]

@ashhart @javierbits Here is the 27B arm you asked for in #2941. Dense 27B gives no clear signal. Its noise between servers is about 0.8 % at 32K, larger than the 0.5 % gain the 35B showed, and the tokens are equal everywhere.

Prompt	base TTFT median [min-max]	head TTFT median [min-max]	paired head - base, per round	adjacent pairs slower
2,048	1.5708 s [1.5675-1.5778]	1.5709 s [1.5659-1.5800]	-1.8 / +0.3 / -0.5 ms	3 of 6
8,192	6.4181 s [6.3972-6.4275]	6.4034 s [6.3932-6.5353]	+58.6 / +0.6 / -24.0 ms	2 of 6
32,768	27.7425 s [27.6491-27.8814]	27.6764 s [27.6459-27.8359]	+134.9 / -143.9 / -104.4 ms	2 of 6

• At 32K the adjacent pairs have a median of -70.5 ms, but round 1 went +135 ms. 8K is level, with one +121 ms outlier pair. 2K is level.
• Read it as "no loss shown", not as a gain. The 35B rows stay the evidence for the ledger number.
• token_sha256/output_sha256 are equal across all 12 servers at each length: adcd6d20…/8bde9a13…, eabb9ee4…/bfda9f68…, bfdffc1b…/3e1bd53e….

Setup, the same as the 35B run:

• Binaries: base a3aecd8d0 md5 1375faa0…, head e7c385e96 md5 a479e68f….
• The current head e34a57994 only adds the docs commit on top of ce33fd6a7, which I checked against the measured head before.
• Order: base, head, head, base for 3 rounds, with a fresh server per arm, the GPU otherwise idle, the clock capped at 2300 MHz, and the page cache dropped before each start.
• Model: a Qwen3.8-27B NVFP4 checkpoint with an NVFP4 head (21.19 GiB at load). Flags: --parallel 1 --context 40960 --no-drafts --prompt-cache-gib 0.
• Prompts: your gate.py cold, 16 greedy tokens.

[ashhart]

Thanks @javierbits, and thanks @jschmied for the quiet-GB10 comparison. The launch trace, sampled/default-drafter rows, prompt-version note, removed key bit and measured ledger entries address those points. Two items remain:

1. zig/src/families/qwen3_5/cuda_engine.zig:382: on current main this becomes 602 lines, adding one lean finding. Move a small helper out or make equivalent room so it stays at most 600, with zero new findings over main's 341. Refresh the description's counts too.
2. docs/receipts/qwen-unread-chunks-2941/README.md:78: add the requested main/head byte or hash comparisons for all recurrent/conv state and K/V at the 8,192- and 16,384-token cache marks, beside the existing reply hashes and cached-token counts.

• Synthena (Ash's Agent)