[jschmied]

What this changes

#2444 P3 step F2, built as ashhart answered the two open questions (comment 6764448869): whole-layer pieces first, and
the draft's two buffers wider than a page as a lane's own block, charged as the lane's fixed bytes and shown in the
load report.

Today a Flash Next pool lane (TF_FLASHNEXT_KV_PAGES) claims one consecutive run of pages for its recurrent states and
its MTP head's buffers (48 pages of 256 rows with the head and pending rows, 16 without the head). A pool with no free
run that long refuses the lane even when it has the pages. With TF_FLASHNEXT_STATE_PAGES=1 (only with
TF_FLASHNEXT_KV_PAGES; refused by name otherwise) the lane takes any free pages from its promise instead and places
each buffer as one piece inside one page:

• the stream's own state: 36 DeltaNet states (3 MiB each), 36 conv tails, the n-gram tail (73 pieces);
• the head's: 36 spares, 36 saved conv windows, 36 saved projection blocks, the saved n-gram tail and rows, its output
  row, the draft head's scores, best values, and with pending rows (the default) the log's control word and 36 layer
  logs (150 pieces).

The head's streams (31.5 MB at 1536-row chunks) and window logits (15.9 MB) are single matrices wider than a page.
They are one block a lane, allocated at load outside the pool. The block is in the layout's fixed_bytes, so the
load plan charges it to every lane, and the load report names it.

No kernel changes. Every kernel already takes these buffers by pointer, one layer at a time. The only host sites that
reached into the head's saved block by offset (5, in cuda_mtp_ops/cuda_mtp_roll) now read their own pointers
saved_tail/saved_nrow, and the rollback graph key hashes them.

Unset, nothing changes: the run, the layout and every buffer are as on main.

Core vs family:

part	where
packer: first-fit decreasing, 256-B aligned, each piece inside one page; check refuses overlap / a piece past its page	core: core/kv_pieces.zig (new)
Lease.state (pages beside the token table), Pool.takeState, finish gives them back	core: core/kv_pool.zig
Layout.pieces / statePages() in need, plan, holds, the window cap; Layout.outside (of fixed_bytes, wider than a page)	core: core/kv_pool.zig, native/cuda_kv_pool.zig (2 lines)
load report: pieces, pages a lane, pad; a lane's fixed part and its wide buffers	core: core/kv_groups.zig
host tests: FN counts, determinism, refusals, plants; admission equal to a run of equal pages; fragmentation; LEAK	core: kv_pieces_test.zig, kv_pool_state_test.zig
piece list, layoutPieces, lane open/close/fits, states mapped from the map	family: flashnext/cuda_kv_lanes.zig
Draft.pieceSizes / wideBytes / initPieces, saved_tail/saved_nrow	family: flashnext/cuda_mtp_draft.zig (+ 5 host sites)
switch, refusals in words	family: cuda_lanes_load.zig, cuda_lanes_explain.zig; core/profile.zig, docs/profiles.md

Q3 (Qwen 3.5/3.6, PLAN in #2444) is meant to reuse the core parts as is, giving its own piece list. Metal has no Flash
Next pool, so nothing there to share.

What each lane pads to (256-row pages with the head, 7.52 MiB a page):

	today's run	pieces	wide block outside the pool
head + pending rows (default)	48 consecutive pages	223 pieces on 45 pages	45.2 MiB
head, TF_FLASHNEXT_KEEP_LATE=0	41	186 on 37 (pad 20.6 MiB, 8.0 %)	45.2 MiB
--no-drafts	16 (6.94 MiB pages)	73 on 18	none

• Issue: Refs Roadmap: one paged KV pool and a radix prompt cache for every family #2444 (P3, F2).
• Stacked on: main 6c5cef8 (rebased 2026-11-12, as ashhart asked in Roadmap: one paged KV pool and a radix prompt cache for every family #2444 comment 6824699204). flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992 builds (c) on this
  head and the two land together.
• The rebase was clean, with no conflict. The first commit is now qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's copy of the shared core commit (now fac7b74, same
  patch-id), as asked in 6825628750, so flashnext: pool lanes keep their states and draft as pieces on whole pages, no run (#2444 P3, F2) #2898, flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992 and qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939 carry the same commit. Against this PR's old first commit it
  adds the reworded kv_pieces_test.zig header, Q3's takeState fix (highest free ids, the free stack's order) and
  qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's spare-lane line in native/cuda_kv_pool.zig served (off for Flash Next, which sets neither all_on_pages nor
  engine_paged). The two Flash Next commits have the same added and removed lines as on 1ed15fd; a fourth names the
  smallest page in the PieceOverPage refusal, and a fifth moves the switch's environment row to main's flashnext scope.
• Overlap: open PRs on the same files: qwen3_5: TF_QWEN_KV_HOST - pinned-host KV pages (capacity mode for long windows) #2637, cuda: opt-in host rules for families without prompt states #2710 and cuda: dots.mocr's image tower on the dense engine #2732 (kv_pool.zig / native/cuda_kv_pool.zig) already conflict with
  main; flashnext: refuse a malformed TF_FLASHNEXT_* setting by name before device work #2585 (Flash Next draft/window graph) and prompt radix: checkpoint parts per cache group, one hold a page, page-only stores (#2444 P2, C3) #2875 (P2.1) merge clean with this head.

Checks on this head

• Restacked on main 6c5cef8 (10 commits past 3885dce, none in Flash Next, the KV pool or the environment table) with
  qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's updated core commit fac7b74 (same patch-id 02a90f2d91d9): its only change from 7e2957d is the TooMany text
  in native/cuda_kv_pool.zig, "state pages" for "state runs", and its test. kv_pool.zig, kv_pool_state_test.zig and
  cuda_kv_pool.zig are byte-equal to qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's head 250994a.
• Head commit: 59ba8b4 (five commits on main 6c5cef8). Before the rebase it was 1ed15fd on 33ada77.
• x86_64 CI box on 59ba8b4: zig build -Dfixed-cpu test test-golden test-kernel-contract test-server-security test-pull-security fuzz-cortheon check-release-paths: 724/724 steps. zig build check-native-cuda-host compile-cuda-tests: 16/16. python3 tools/pr_check.py: 0 errors, lean 345 vs 345 (0 new); its one warning is the CHANGELOG line, which is in this body for
  ashhart to place. zig fmt --check, git diff --check: clean.
• GB10 host on 59ba8b4: zig build test-kv-pool test-cuda-host: 396/396 steps.
• New host tests: kv_pieces_test.zig (Flash Next counts 186/37 and 223/45 at 256-row pages, determinism, a piece wider
  than a page refused by name, overlap/past-page maps refused); kv_pool_state_test.zig (need/plan/holds equal to a run of
  equal pages, fixed part counted at admission, takeState/finish, fragmentation: every other page held, claimRun refuses,
  takeState admits; LEAK: failed state allocation, cancel after open, failed table write: census back to load);
  cuda_kv_pieces_test.zig (fake device: a fragmented pool opens a pieces lane and refuses a run lane, two lanes with the
  head place every piece inside their own state pages with no overlap, failed upload/fill leave the pool as at load, a
  layout from another packing refused); the switch is exact 0 or 1, refused by name when Flash Next loads, before device work: a flashnext scope row, so a
  malformed value no longer stops other families' CUDA starts (environment_test; the scope's count test and
  docs/environment.md move to 30 settings, 21 switches).
• Planted controls, scratch copies, red: packer places overlapping pieces (kv_pieces.zig:64, 3 tests red); conv tails
  mapped onto the state pieces (cuda_kv_lanes.zig mapStates, 2 tests red).
• core/kv_pool.zig and kv_pool_state_test.zig are byte-equal to qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's head f4a0bdf (cmp), takeState fix included.
• PieceOverPage (cuda_lanes_explain.zig): the load logs the page's bytes, the largest piece and the smallest
  TF_FLASHNEXT_KV_PAGES that holds it, computed from the same piece sizes the packer places (smallestRows in
  cuda_kv_lanes.zig): 104 rows with the head, 112 without. The refusal points at that line, as Qwen's does. Host test:
  both values, the smallest page loads, one step less refuses. Planted red: smallestRows one step short (cuda_kv_pieces_test red by name, "expected 104, found 100", 1 failed).

Served gates (GB10, Qwen3.8-Flash-Next-FP4, --context 16384 --parallel 4)

The rows in this section are from the old head 1ed15fd against main 33ada77. The pair lands as one, so the served rows
for the rebased code are on #2992's head, which contains this one: its kit and A-B-B-A were rerun after the rebase. The
"don't land yet" below is about this PR alone, and #2992's (c) answers it: there main and pieces admit the same streams at
76-79 GB.

Conformance kit (tools/conformance.py, flashnext-cuda profile with cancel/evict blocks), on this head, page cache
dropped before every server: pages off, TF_FLASHNEXT_KV_PAGES=256, and 256 + TF_FLASHNEXT_STATE_PAGES=1. All 7 gates
pass in each: drafted == plain 10/10 (448 of 724 MTP drafts accepted), repeat 10/10, concurrent == solo 76/76 at 1/2/4/8
(drafted and plain, greedy and seeded), resumed == fresh 6/6, refusals 4/4, cancel 4/4 (censuses back to idle), evict 3/3.
Every kit row's token sha against main (33ada77) in the same paging: 126 rows, 114 keys, 0 differ, for off, 256, and
256 with pieces (on == off == main).

Load line with pieces (256): state, in the pool as pieces: 223 pieces, 312.22 MiB on 45 pages a lane, 25.98 MiB padding (7.7 %); a lane's fixed part, outside the pool and counted at admission: 45.16 MiB, 45.16 MiB of it buffers wider than a page. Main: state run, in the pool: 357.39 MiB on 48 pages. A lane: 383.4 MiB vs 360.8 MiB (+22.6 MiB, +6.3 %).

Speed, A-B-B-A twice per paging, fresh server each arm (f2 ab-table):

cell	c=1 tok/s	c=4 tok/s	TTFT 8.8k
256: main vs pieces	+0.08 % (rounds -0.32 / +0.48)	-0.09 % (-0.09 / -0.09, ranges overlap)	+0.04 %
off: main vs branch	-0.18 %, rerun -0.26 %	-0.06 %, rerun -0.03 %	+0.43 %, rerun -0.30 %

Admitted streams at a memory limit (TENSORFOLD_CUDA_MEMORY_LIMIT_GB, --parallel auto): lanes equal at 76-79 GB, but the
pool is 36-42 pages smaller (every lane's 45 MiB block, idle lanes too) and a stream saves only 3 pages. Concurrent
16k-window streams: 5 vs 4 at 77 GB (equal at the other five limits); 4k-token streams: 6/7/8 vs 5/6/7 at 76/76.5/77 GB.
Don't land yet: the idle wide block costs a stream in a measured cell, which is the trigger for (c) (streams and window
scores read by row, so they fit pages too) or for a block made only while a lane is busy.

Receipt

• Served binaries (old head): main 33ada77 tensorfold-native md5 5c6f97bb, 1ed15fd md5 85868fe8 (zig build native -Dsm=121).
• New switch: TF_FLASHNEXT_STATE_PAGES (exact 0 or 1; 1 needs TF_FLASHNEXT_KV_PAGES), off by default.
• Environment: GB10, Zig 0.17.0, nvcc 13.x, clock-capped; x86_64 CI box.
• Raw logs: ledger kv-p3/f2 (runs/conf, conf2, conf3, ab, ab-off2, fit; plants/).

Proposed CHANGELOG line (Unreleased, for the maintainer to place):
- Flash Next: TF_FLASHNEXT_STATE_PAGES=1 keeps a pool lane's states and MTP buffers as pieces on any pages of its lease, so a fragmented pool still admits it; the head's two wide buffers are a lane's fixed bytes (#2444).

[ashhart]

Thanks @jschmied. F2 reads well. The packer is small and deterministic and refuses a bad map by name, Lease.state and takeState follow take closely, and statePages stands in for the run everywhere it was counted: need, plan, holds, held and the CUDA budget. With the switch unset, the one change on the served path is that the saved copies and the rollback read saved_tail and saved_nrow, which every existing constructor sets to the same addresses as before (in point), and your kit shows 0 of 114 keys differing. Holding this for (c) once the memory-limit table showed the idle wide block costing a stream was the right call.

We checked 1ed15fd: CI is green, lean has 0 new problems, no file passes 600 lines (kv_pool.zig is at 595), it merges cleanly with today's main, and test-kv-pool (70 of 70), the environment tests, Flash Next's host tests (258 of 258, your five new ones included) and the Linux CUDA cross-compile all pass on our side. Two small things:

1. zig/src/core/kv_pieces_test.zig:1 and :7 still cite f2_pack.py and fn/cuda_kv_lanes.zig pieceSizes. The script isn't in the tree, and pieceSizes is in cuda_mtp_draft.zig. qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's copy of the shared core commit already has the reworded header. Please take it here and in flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992, so both stacks carry the same commit.
2. zig/src/families/flashnext/cuda_lanes_explain.zig:25: the PieceOverPage text says pages need 128 rows or more, but the floor is lower and depends on the head. With the head, a 104-row page already holds a 3 MiB state; without it, a 112-row page does. Please compute the smallest page that holds the largest piece and name it in the refusal, as Q3's load line does.

As agreed on #2444, this stays a draft and lands with #2992 in one train. #2992 already carries Q3's takeState (highest free ids), so the train gets it. We'll review #2992 next. The cell to land on is your fit table with (c) in, plus the kit and speed rows on the train's head.

• Synthena (Ash's Agent)

[jschmied]

@ashhart both done, on both stacks:

1. The first commit of flashnext: pool lanes keep their states and draft as pieces on whole pages, no run (#2444 P3, F2) #2898 and flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992 is now qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's copy of the shared core commit (7e2957d, same patch-id), so all three PRs carry the same commit. It brings the reworded kv_pieces_test.zig header with no tool or wrong-file references. It also brings Q3's takeState fix and qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's spare-lane line in cuda_kv_pool.zig served, which is off for Flash Next (it sets neither all_on_pages nor engine_paged). kv_pool.zig and kv_pool_state_test.zig are byte-equal to f4a0bdf. flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992's former takeState commit is gone, because its content is already in the first commit.
2. PieceOverPage now names the floor. smallestRows computes the smallest TF_FLASHNEXT_KV_PAGES whose page holds the largest piece, using the same sizes the packer places: 104 rows with the head and 112 without, as you said. The load logs the page's bytes, the largest piece and that size, and the refusal points at that line, as Qwen's does. A host test checks both values, that the smallest page loads and that one step less refuses. Planted control: smallestRows one step short goes red by name ("expected 104, found 100").

Heads: #2898 d432157, #2992 25108af, both on main 3885dce. Host test-kv-pool test-cuda-host 392/392 on each; x86 720/720 + 16/16, lean 0 new.

[jschmied]

@ashhart both stacks are restacked on main 6c5cef8, with #2939's updated core commit fac7b74 as their first commit. It has the same patch-id as #2939's (02a90f2d91d9), so #2898, #2992 and #2939 still carry the same commit. Its only change from the previous copy is #2939's point 4: the TooMany text in native/cuda_kv_pool.zig says "state pages" instead of "state runs", in both the message and its test. kv_pool.zig, kv_pool_state_test.zig and cuda_kv_pool.zig are byte-equal to #2939's head 250994a. Heads: #2898 59ba8b4, #2992 f0fe4b7. Every other file has the same added and removed lines as before, and main's 10 new commits touch no Flash Next, KV pool or environment file, so the kit, fit and A-B-B-A rows carry over. Gates on both heads: host test-kv-pool test-cuda-host 396/396 each; x86 724/724 + 16/16, pr_check 0 errors, lean 345 vs 345 (0 new), fmt and diff --check clean.

[ashhart]

Thanks @jschmied. The restack reads right:

• the first commit on flashnext: pool lanes keep their states and draft as pieces on whole pages, no run (#2444 P3, F2) #2898 and flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992 carries the same files as qwen3_5 cuda: a pool lane's state as pieces on any whole pages of its lease, no run (#2444 P3, Q3) #2939's fac7b7422c;
• the TooMany text says "state pages";
• PieceOverPage now points at the load line, which names the smallest page that holds a lane's largest piece (104 rows with the head, 112 without), with a host test for both;
• CI is green on flashnext: pool lanes keep their states and draft as pieces on whole pages, no run (#2444 P3, F2) #2898's 59ba8b498 and on flashnext: the MTP head's streams by row through the lease's pages, no per-lane block outside the pool (#2444 P3, F2 part c) #2992's f0fe4b77c.

We have no more code asks on either PR.

Before the two land together, we'll run one cell on a GB10 on #2992's head against main 6c5cef894. That's Qwen3.8-Flash-Next-FP4 with the defaults, and again with TF_FLASHNEXT_KV_PAGES=256 and TF_FLASHNEXT_STATE_PAGES=1, comparing token hashes and speed. If it matches your rows, #2898 and #2992 land as one train.

#2992 removes the per-lane block that this body's line describes, so this is the CHANGELOG line we'd use for the pair:

- Flash Next on CUDA: TF_FLASHNEXT_STATE_PAGES=1 keeps a pool lane's recurrent states and MTP head buffers as pieces on any pages of its lease; the head reads its streams by row through those pages, and a draft's draws read the window scores where its verify wrote them, so a fragmented pool still admits a lane and no per-lane block sits outside the pool; off by default, and tokens are unchanged (#2898, #2992, part of #2444). Thanks to @jschmied.

Tell us if you'd like it worded differently.

• Synthena (Ash's Agent)