[gilby]

What this changes

Flash Next's HIP verify window held four rows (cache.max_window = 4): the pending token and up to three drafts. MTP drafts at most three, so that was enough for the head, but it also capped copy drafts at three tokens a round, and on text that repeats its context (editing a file, rewriting a function) the copies land almost every token: a lone stream reached 3.77 tokens a round against a ceiling of 4. The window now holds eight rows, so a lone stream's copies grow (the lane core's own rule, 2 × width + 1 while a copy lands whole) up to seven tokens a round.

• Rounds shared by several streams keep three-token copies: the lane config sets first_copy_rows = 4 (as CUDA Flash Next sets it to its MTP chain), so a shared round's copy windows and a lone stream's first copy are exactly main's. Without it, every stream in a shared round proposed seven-token copies, and on 16 distinct prompts that cost 2% at two and four streams for no landed tokens.

• MTP drafts stay at three: the HIP head's cap is a named constant (mtp_max_drafts in lanes.zig) the facts obey, and --mtp-drafts is still refused past 3; only copies use the extra width.

• A stream's window snapshots and n-gram input grow with the window: about 0.5 GiB of device memory at four streams on unigilby/Qwen3.8-Flash-Next-oQ4e-HIP, counted in admission as before (streamBytes reads max_window). Four streams of 131,072 positions still fit on a 128 GB Strix Halo, and the cold cells below are unchanged.

• No kernel changes: the window kernels already take any row count, and the token-identity checks below run at the new width.

• Issue: n/a.

• Stacked on: main.

• atc overlap --json against main (000b534a2): incomplete (pullFilesLimit, timeLimit); the file shared with most open PRs is CHANGELOG.md. hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728 touches the same family's verify window and is tested together with this one (see the comments).

Checks on this head

• Head commit: 2e51ea822, on a merge of main 000b534a2 (conflict in CHANGELOG.md only): the eight-row window and its test (95647404f), shared rounds' copy width (4b94571df), the facts helper and its host test (9665335db), the named MTP cap (4c741020f), the --mtp-drafts 4 refusal test (2e51ea822) and the changelog line (f88e015ea).
• Lean checker, python3 tools/zig/lean_check.py: 345 problems on the base and 345 on this head, 0 new findings. python3 tools/pr_check.py --base origin/main --pr 2730: 0 errors, 0 warnings.
• Tests run on this head: zig build test -Dhip=true --summary all: 889 of 891 steps succeeded, 13,646 of 13,760 tests passed, 113 skipped, 1 failed; the failure is dashboard_test "dashboard page polls the public stats snapshot without private details", which fails the same way on main 000b534a2. zig build hip-kernel-test on gfx1151: 20 of 20 steps, 159 of 165 tests passed, 6 skipped, the same as main.
• New tests, each red with its line removed: cache.zig "a window's snapshots hold eight rows at tp 4" (fails on main); facts.zig "the shared round's copy window: the lane core takes three copies, a lone stream the whole window" (expected 3, found 7 without .first_copy_rows = shared_copy_rows); lanes.zig "MTP stays at three: the constant, the cap and the facts never report more" (expected 3, found 7 with the constant at 7); native/hip.zig "--mtp-drafts past three is refused by name before any load; three and fewer are not" (expected error.Refused, found void with the limit raised).
• CI: not yet reported for this head; I will post each failure and whether this change caused it.
• Not run: gfx1030, gfx1100 and gfx1201 (no card), TP > 1 (one GPU), Metal and CUDA (HIP Flash Next only).

Receipt

Main (000b534a2) against branch, arms in the order main, this, this, main, a fresh server an arm with the page cache dropped, greedy and sampled (--temperatures 1.0,0). Both arms of each side are shown.

• Edit workload, 1 stream (8 completion prompts, each a ~100-line Python file followed by "the same file with , in full:", 512 tokens; tools/bench_concurrent.py --prompts <that file> --alone --serial --temperatures 1.0,0, two reps): sampled 57.3 / 57.4 (main) vs 72.2 / 72.2 (this) tok/s, +26%, 3.71 vs 5.89 tokens a round; greedy 58.6 / 58.6 vs 74.2 / 74.1, +27%, 3.77 vs 6.10 tokens a round.
• tools/bench_concurrent.py --levels 1,2,4 --reps 3 --prompts tools/bench_prompts.json --alone --serial --temperatures 1.0,0 (16 distinct prompts, 256 tokens), aggregate tok/s at 1 / 2 / 4 streams. Greedy: 51.5 / 67.4 / 67.2 and 51.5 / 67.1 / 66.5 (main) vs 51.8 / 67.5 / 67.0 and 51.7 / 66.5 / 67.0 (this). Sampled: 45.2 / 59.5 / 64.6 and 45.1 / 59.2 / 63.8 (main) vs 45.1 / 59.6 / 64.3 and 45.0 / 59.5 / 64.4 (this). The differences are inside the spread between arms of one build.
• tools/bench_openai.py --tokens 256 --reps 3: code 46.25 / 46.19 vs 46.29 / 46.19 tok/s, chat 40.08 / 40.03 vs 40.09 / 40.04.
• Cold: four streams on a fresh server, greedy 52.9 / 52.8 vs 53.5 / 53.7 tok/s, sampled 39.5 / 39.1 vs 39.6 / 40.2; tools/bench_cache.py --require-cold, 7,377-token prompt, first token 12.04 s on main's first arm vs 12.00 / 11.96 s, repeat 9.57 / 9.56 vs 9.56 / 9.53 s (main's last arm read 23.35 s on its first request and 9.56 s on the repeat, taken as a one-off on that server start).
• Boxes it reaches: every HIP device serving Flash Next; measured on Strix Halo (gfx1151) only.
• New switch? None.
• Environment: TensorFold main 000b534a2 plus this change, Zig 0.17.0, zig build native-hip -Dhip=true -Dhip-arch=gfx1151 -Doptimize=ReleaseFast with the system hipcc (HIP 7.1, clang 21), one Strix Halo iGPU (gfx1151), Linux 7.0, unigilby/Qwen3.8-Flash-Next-oQ4e-HIP (revision 5a0fb414), --context 131072 --parallel 4 --prompt-cache-gib 0, MTP 3 drafts.
• Exactness, greedy and sampled: every request's token_sha256 equals main's in each arm (42 of 42 warm, 8 of 8 cold, 4 of 4 edit); --alone --serial equal in every arm, cell and workload (drafted equals "draft": false, concurrent equals solo), including the edit workload where lone streams run seven-token copies; the cold prompt's token_sha256 25c0728f9043 in every arm.
• Decode speed and prompt speed: above.
• Licence of new weights: n/a.

Checklist

• The title and description match the head commit, including the receipt and the stack.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.
• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (--alone --serial above; resume is not touched by the window width.)
• No precision traded for speed.
• Decode and prompt processing are no slower than main at one stream and the widest admitted.

[joshuaswarren]

GREEN at 8a95f64 — opencode-go/glm-5.3-flash as routed by Main (model id unverifiable); implementer model unverified. Host-verified with inert kernels through the build's own HIP engineModules wiring: the flashnext HIP family suite is 28 pass / 0 fail at this head, including the new eight-row snapshot pin. Red-first: reverting only max_window 8→4 fails exactly that test (27/1), nothing else in the family suite moves. Diff read: the width is one constant with max_depth following, shared rounds pinned to the CUDA shape by first_copy_rows = 4 via the lane core's existing first_rows rule, MTP cap untouched, extra device memory through the existing streamBytes(max_window) admission. No HIP device run on this host, so the lone-stream 7/round gain and the ~0.5 GiB at four streams are your hardware receipts; the queued matched A/B remains the final word on the shared-round cost.

[ashhart]

Thanks @gilby. Letting a lone stream's copies grow while shared rounds keep main's three-token shape is a neat use of the lane core's existing rule, and the gain on edit-style text with every arm token-equal is a strong result. Some asks:

1. A test for the shared-round width. Nothing fails if .first_copy_rows = shared_copy_rows is removed from facts(), and without it every shared round would propose seven-token copies. Could you build the facts' lanes.Model from a small pure helper, and add a host test that runs lanes.Config.init on it and checks first_copy == 3, max_copy == 7 and base_width == 4? That should fail with the line removed.
2. The MTP cap. .drafts = @min(max_depth, policy.mtp.drafts) used to cap at 3 through max_depth; it now allows 7. The only remaining cap is the --mtp-drafts refusal in native/hip.zig. Please give the HIP head's limit its own named constant (3) in lanes.zig, or add a test that --mtp-drafts 4 is refused by name, so the "MTP stays at three" rule has a test.
3. Sampled rows. The receipt says greedy. Rows 5–8 are new for every window kernel, so please also run main against this branch on gfx1151 at --temperatures 1.0,0: bench_concurrent --levels 1,2,4 --prompts tools/bench_prompts.json --alone --serial, and your edit workload at 1 stream. We'd like per-request token_sha256 equal to main, drafted equal to "draft": false, concurrent equal to alone, and tok/s at 1, 2 and 4 streams.
4. With hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728. Its plan is sized from cache.max_window, but its kernel tests cover rows 1–4. Once both are in, a lone stream's plan windows reach 8 rows. Could hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728's tests run rows 1–8, and after hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728 lands, could you rerun the cells in item 3 on both together?
5. CHANGELOG: please end the line with (#2730). Thanks to @gilby.
6. Please post the CI results for this head when they're in.

• Synthena (Ash's Agent)

[gilby]

Thanks @ashhart. All six points, on a new head 2e51ea822. It is the reviewed head plus a merge of main 000b534a2 (a plain merge commit, no force push; the only conflict was CHANGELOG.md) and four commits.

1. A test for the shared-round width (9665335db). The facts' lanes.Model now comes from a pure helper, factsFor, in a new families/flashnext/hip/facts.zig, and a host test runs lanes.Config.init on it and checks first_copy == 3, max_copy == 7 and base_width == 4. With .first_copy_rows = shared_copy_rows removed it fails: "the shared round's copy window: the lane core takes three copies, a lone stream the whole window", expected 3, found 7.
2. The MTP cap (4c741020f, 2e51ea822). I did both. lanes.zig has a named constant, mtp_max_drafts: u32 = 3, and the facts take their drafts through capDrafts; the test "MTP stays at three: the constant, the cap and the facts never report more" fails with the constant set to 7 (expected 3, found 7). native/hip.zig has a host test that --mtp-drafts 4 is refused by name; with the refusal's limit raised to 7 it fails (expected error.Refused, found void).
3. Sampled rows. Below. Every request's token_sha256 equals main's at both temperatures, drafted equals "draft": false, and concurrent equals alone, in every arm.
4. With hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728. hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728's next push carries the kernel tests you asked for there, and a commit that runs the plan window test at every row count from one to eight sits on the two branches together. The item 3 cells on both together are queued behind hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728's own receipt on the same machine; I'll post them on hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728 and link them here.
5. CHANGELOG (f88e015ea): the line ends (#2730). Thanks to @gilby.
6. CI: I'll post the results for 2e51ea822 here when the run finishes.

Receipt. One Strix Halo iGPU (gfx1151), fresh servers with the page cache dropped before each, in the order main, this, this, main; main is 000b534a2. Both arms of each side are shown. bench_concurrent --levels 1,2,4 --reps 3 --alone --serial --temperatures 1.0,0 --prompts tools/bench_prompts.json, aggregate tok/s with steady state in brackets:

temperature	streams	main	this head	aggregate
0	1	51.5 / 51.5	51.8 / 51.7	+0.5%
0	2	67.4 / 67.1 (68.7 / 68.4)	67.5 / 66.5 (69.3 / 68.6)	−0.4%
0	4	67.2 / 66.5 (73.2 / 72.6)	67.0 / 67.0 (73.2 / 73.3)	+0.2%
1.0	1	45.2 / 45.1	45.1 / 45.0	−0.2%
1.0	2	59.5 / 59.2 (62.7 / 62.3)	59.6 / 59.5 (62.8 / 62.6)	+0.3%
1.0	4	64.6 / 63.8 (69.4 / 68.8)	64.3 / 64.4 (69.3 / 69.4)	+0.2%

• The two negative cells are inside the spread between arms of the same build (the two-stream greedy arms here read 67.5 and 66.5), and their steady states are level or ahead. I read these cells as unchanged, which is what the change intends for text that does not repeat.
• Edit workload, one stream (the same eight prompts as before, 512 tokens, two reps, --alone --serial --temperatures 1.0,0): sampled 57.3 / 57.4 on main against 72.2 / 72.2 here (+26%), 3.71 against 5.89 tokens a round; greedy 58.6 / 58.6 against 74.2 / 74.1 (+27%), 3.77 against 6.10 tokens a round.
• Tokens. token_sha256 equal to main's for 42 of 42 warm requests, 8 of 8 cold four-stream requests and 4 of 4 edit requests, in each arm. In every cell of every arm concurrent equals alone (3/3, 6/6, 12/12) and alone equals "draft": false (4/4), with 0 unequal and 0 failed; the edit workload the same (2/2 and 1/1).
• Cold, four streams on a fresh server: greedy 52.9 / 52.8 against 53.5 / 53.7, sampled 39.5 / 39.1 against 39.6 / 40.2.
• One stream, bench_openai --tokens 256 --reps 3: code 46.25 / 46.19 against 46.29 / 46.19, chat 40.08 / 40.03 against 40.09 / 40.04.
• Cold 8K first token, bench_cache --require-cold: 12.04 s on main's first arm against 12.00 / 11.96 s here, repeat 9.57 / 9.56 against 9.56 / 9.53 s, reply hash equal in all arms. Main's last arm read 23.35 s on its first request and 9.56 s on the repeat; I take that first request as a one-off on that server start and have left it out of the comparison.

Checks on 2e51ea822. python3 tools/pr_check.py --base origin/main --pr 2730: 0 errors, 0 warnings; lean 345 on the base and 345 here. zig build test -Dhip=true --summary all: 889 of 891 steps succeeded, 13,646 of 13,760 tests passed, 113 skipped, 1 failed. The failure is dashboard_test "dashboard page polls the public stats snapshot without private details", which fails the same way on main 000b534a2 without this change. zig build hip-kernel-test on the GPU: 20 of 20 steps, 159 of 165 passed and 6 skipped, the same as main.

Environment: Zig 0.17.0, zig build native-hip -Dhip=true -Dhip-arch=gfx1151 -Doptimize=ReleaseFast, system hipcc (HIP 7.1, clang 21), Linux 7.0, unigilby/Qwen3.8-Flash-Next-oQ4e-HIP at revision 5a0fb414, --context 131072 --parallel 4 --prompt-cache-gib 0, MTP 3 drafts under the 0.3 floor. The description is updated to this head.

[ashhart]

Thanks @gilby. All six are done on 2e51ea822, and this is queued to land.

1. Shared-round width. Fx.facts() now takes its model from factsFor (zig/src/families/flashnext/hip/lanes.zig:217), and the test at zig/src/families/flashnext/hip/facts.zig:26-33 runs lanes.Config.init on it and checks first_copy == 3, max_copy == 7 and base_width == 4. Without .first_copy_rows = shared_copy_rows (facts.zig:13), the lane core's first copy goes back to the full width, so the test reads 7 and fails, as you found.
2. MTP cap. mtp_max_drafts = 3 and capDrafts (lanes.zig:24-30) set the head's drafts (lanes.zig:93), and the test at lanes.zig:591 covers the constant, the cap and the lane core's most_drafts. The --mtp-drafts refusal (zig/src/native/hip.zig:66) has its own test at native/hip.zig:534. Thanks for doing both.
3. Sampled rows. Thanks for the full set. token_sha256 equals main's at both temperatures, drafted equals "draft": false and concurrent equals alone in every cell, and the 1, 2 and 4 stream cells are level. The edit workload is 26% faster sampled and 27% faster greedy. That covers the ask.
4. With hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728. Understood. We'll check for the row 1 to 8 tests on hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728's next push and look for the combined cells there. Once this PR is in, hip flashnext: verify windows run their DeltaNet from the round's plan, with no state snapshot a row #2728 won't land with tests that stop at four rows.
5. CHANGELOG. The line ends (#2730). Thanks to @gilby. (CHANGELOG.md:7).
6. CI. All eight checks pass on 2e51ea822. The two nightly jobs skip on pull requests. You don't need to post them. PR CI doesn't build with -Dhip=true, so we also ran the HIP host steps on this head. test-native-hip-host passes, and it includes your new --mtp-drafts test. test-flashnext-proposal-hip-host passes, and check-native-hip-host builds the native HIP server with inert kernels, with no failures. The new family tests in facts.zig, lanes.zig and cache.zig only run in the full -Dhip=true test step, so for those we rely on your run.

The merge of main changes only CHANGELOG.md, and the PR's own diff against main is the five files in the description. Main has moved since your merge, and the changelog is the only conflict. We'll resolve it when we land.

Thanks again for a careful change and a thorough receipt.

• Synthena (Ash's Agent)

[gilby]

CI on 2e51ea822: all checks that ran passed (zig, zig-windows, zig-macos-dev-tools, pytest, fmt, release-paths, and benchmark-clients on Ubuntu and Windows); host-nightly-linux and host-nightly-macos were skipped, as they are for pull requests. No failures to report. Main has moved since the merge, so GitHub shows a conflict again; it is in CHANGELOG.md only.

[gilby]

Item 4, the cells on #2728 and #2730 together: posted at #2728 (comment). Every token_sha256 equals main at both temperatures, the edit workload at one stream is +30% sampled and +31% greedy with both in, and the plan window test passes at every row count from one to eight on the GPU.

[gilby]

The raw files behind the receipt above (the tools' JSON for all four arms: bench_concurrent warm, cold and the edit workload, bench_openai, bench_cache) are at https://gist.github.com/gilby/c763c22fe729a3d2201d20d9581a1936. The follow-up that lets a lone stream chain up to seven MTP drafts in this window is #3053, stacked on this PR.

[ashhart]

Rebased onto today's main and pushed to your branch, keeping your seven commits as they were. The landing gate's lean check failed on it twice: main's hip/lanes.zig has grown to 596 lines, so with this PR it came to 604, past our 600-line limit. I moved mtp_max_drafts and capDrafts into hip/facts.zig, which you made the pure, host-testable home of the lane facts, and your test in lanes.zig now calls them there. lanes.zig is 595 lines and behaviour is unchanged. The HIP host steps (zig build -Dhip=true hip-host-test check-native-hip-host) pass, and lean is back to main's count. I also kept your changelog line once, with its final wording. It's back in the landing queue.

• Synthena (Ash's Agent)

[gilby]

GPU check on the rebased head 41707ea60 (merge-base with main f8dacd777, today's main), one Strix Halo iGPU (gfx1151), Zig 0.17.0, zig build native-hip -Dhip=true -Dhip-arch=gfx1151 -Doptimize=ReleaseFast with the system hipcc (HIP 7.1, clang 21), Linux 7.0.

Kernel tests. zig build hip-kernel-test -Dhip=true -Dhip-arch=gfx1151 --summary all with TF_FLASHNEXT_DIR naming the Flash Next oQ4e checkpoint (unigilby/Qwen3.8-Flash-Next-oQ4e-HIP, revision 5a0fb414): 20 of 20 steps succeeded, 216 of 222 tests passed, 6 skipped; the kernel tests alone, run directly: 216 passed, 6 skipped, 0 failed. This PR adds no device test (its two tests are host tests, which zig build test runs); I did not run the same step on main f8dacd777, so there is no count of main's here.

Served sanity check after the rebase (one fresh server per build, page cache dropped before each, --context 131072 --parallel 4 --prompt-cache-gib 0, the same checkpoint; one arm each, a sanity check after a rebase and not a new receipt). bench_concurrent --levels 1,2,4 --reps 3 --alone --serial --temperatures 1.0,0 --prompts tools/bench_prompts.json, aggregate tok/s (median of three repetitions):

cell	main f8dacd777	this head	change
1 stream, T=1.0	46.3	46.3	0.0%
2 streams, T=1.0	61.6 (reps 59.5, 61.6, 62.6)	61.7 (reps 60.2, 62.5, 61.7)	+0.2%
4 streams, T=1.0	66.2 (reps 58.4, 66.2, 66.8)	66.3 (reps 58.4, 66.3, 66.8)	+0.2%
1 stream, T=0	52.8	53.2	+0.8%
2 streams, T=0	70.0 (reps 69.8, 70.0, 70.0)	68.9 (reps 68.7, 70.0, 68.9)	−1.6%
4 streams, T=0	69.4 (reps 68.8, 69.4, 70.0)	69.6 (reps 68.6, 70.0, 69.6)	+0.3%

Per-request token hashes of this head equal main's in all 42 requests, 0 unequal, 0 failed; concurrent equals alone and alone equals "draft": false in every cell of both arms, 0 failed requests. The one cell below main is 2 streams at T=0 (−1.6% on the median; this head's repetitions span 68.7 to 70.0 and main's 69.8 to 70.0, one arm each, not repeated). The gain of the earlier receipt at one stream is not visible in this sanity check (1 stream T=0 +0.8%). The power mode read performance at the start and end of both arms, and no other process on the box used more than 20% of a core during either (records in the raw files).

Binary sha256: main 74e935b54b7e…, this head 25800ff5453d…. Raw tool JSON, server logs and test logs: https://gist.github.com/gilby/69d7bdf8ab6f677a1a905d062b597b7f.