[Mirrdhyn]

Refs #2897.

A stream served alone with the DFlash2 drafter (--parallel 1) went through LaneHost.lone to XpuLanes.alone, which
verified a fixed 15-node tree every round. At long context 16 verified rows each read the whole context, so a smaller
tree is faster with the same tokens (#2897). This takes that choice through lone_costs, as CUDA's lone round does:

• alone grows the tree to cap = min(XPU_TREE, 15) on the host (drafter_tree.grow: lattice, noise, best-first
  tree, now also each draft's own log-chance), prices it with the calls windows.zig makes for a lone head window (prices,
  allocateDrafts, then probe/trial), except least_rows 2 where windows.zig uses 0, and verifies only the first w drafts (settle). A tree
  of w nodes is the first w pops of the 15-node tree and a child pops after its parent, so the prefix keeps every
  parent and the accepted path is the wide path cut at w.
• Chances come from Landing.chance(sampled, own) x parent, as cuda_draft.zig does (xpu_dflash2.chances), updated
  with observeTree on the verified prefix. They are not on the Drafter vtable: Drafted reads
  vtable.chances only when facts.chances or tree_nodes > 0, and XPU sets neither.
• The whole round (lattice, tree, width, verify, the drafter's taps, synchronized at the end) is timed and goes to the
  stream's depth.ms through Rule.observeCost, then lone_costs.observe. The first round (prefill-to-decode
  switch) and the first round after a cold drafter-context restart in resumeLone are not samples. keep leaves the
  band's calibration in Engine.lone_kept on exit, at the end or at the hand-over to the engine; seed starts from it.
• XPU_TREE is a cap. Below 2 nodes the width is fixed and nothing is learned (as before).
• Facts: lone_cost_tile = 4; probes every 4 rows from row 2 (1, 5, 9, 13, 15 drafts).

What the core lacked, and what this extends (small, and CUDA's behaviour is unchanged):

1. lone_costs.enabled required plain_guard, and Rule.costs wrote the stream's own table only under it. XPU leaves
   plain_guard false so that a shared round is not held at 0 drafts. A Who.lone mark (set by
   lone_costs.marked) now makes Rule.costs use the stream's own table for a lone round priced by lone_costs, and
   enabled no longer requires plain_guard. enabled still requires e.alone, a timed lone_cost_tile, no draft
   confidence, a head plan and p.lanes == null (lone_costs.zig); the caller's live (windows.zig) makes it one
   plan. Shared rounds do not change.
2. probe started at 1 row, i.e. a draft-less window. Without plain_guard no round is drafted with nothing, so it
   starts at 2 rows (CUDA's schedule with plain_guard, tile 16, is unchanged: lone_costs_test.zig still passes).
3. lanes.lone_costs is exported so a native backend can call it.

Two things the lone pricing changes beyond the lone driver, with the tokens unchanged:

• An engine round of a single live stream (a hand-over from the driver, or a request the lone driver refuses through
  lane_host_lone.takes: logprobs, prompt logprobs, grammar, think budget, loop guard, a tool call, drafts off) is now priced from the stream's own times with least_rows 0
  (windows.zig, the live branch), so it can pick a round with no draft on the strength of the start-up price; a
  round with 0 drafts is not recorded as a sample (Rule.observeCost).
• Two clocks write the same keys of the stream's table and do not time the same work: the driver times lattice, tree,
  width, verify and the drafter's taps (XpuLanes.alone); the engine's familyRound times flush, verify and the late
  draft. keep/seed share Engine.lone_kept by context band, so a calibration kept by one is read by the other.
  I did not measure the offset between the two.

Exactness: the width only sets how many drafts are verified; the walk keeps the target's own pick (greedy ties to the
lower id, sampled picks keyed by (seed, position, id)).

Measured on an Arc Pro B70 (main and the branch were built with the same local, uncommitted VRAM-guard patch to
zig/src/xpu/rt.zig, and the runs also set that patch's card variable, which the scripts leave out; receipt docs/receipts/xpu-qwen27-lone-tree-width): the same hash per prompt across the
fixed trees of 15, 7 and 3 nodes, the branch and plain decode (greedy, 34 to 121,022 tokens) and across main and branch
(seeded, 34 and 121,022 tokens). Decode tok/s, branch default against main with XPU_TREE 15 / 7 / 3:

prompt tokens	15	7	3	branch
34	141.23	132.50	91.63	136.14
28,946	29.46	40.24	45.21	46.42
58,543	19.74	28.63	34.31	34.84
121,022	10.63	16.63	20.48	21.64

Branch over the best fixed tree: 0.964 at 34 tokens, 1.027 at 28,946, 1.015 at 58,543 and 1.056 at 121,022. At 34
tokens it is 3.6 % below the best fixed tree (15 nodes): it verifies about 11.7 nodes a round and accepts 4.36 drafts
a round where 15 accept 4.67. I looked for the cause on the same card with a per-round log of the price table, the
choice and the measured time (34 tokens, 4 requests, 190 rounds):

• A round costs about the same at 15 nodes as at 14 (40.4 ms, sigma 1.4 ms, against 40.1 ms a round for 15 fixed); a
  single price is a single timing, 3 % noisy, 5 % for the difference of two, and the table keeps stale prices of
  widths it no longer picks (44.6 to 44.9 ms at 11 and 12 nodes against 39.8 at 14).
• The acceptance model is not at fault: 5.46 tokens a round predicted at the chosen width, 5.49 observed.
• The gap is 2 to 3 rounds (47 to 48 against 45) at the same ms a round.

I tried three rules that take the widest width priced within a band of the best one (receipt, "Why 34 tokens stays
below", with the raw runs): a band of 5 % of tokens a ms, and bands of 2 and 4 scatters of the stream's own times. The
5 % and the 4 sigma are bands of the rule, not losses. Greedy tok/s at 34 / 28,946 tokens, against 135.82 / 46.61 for
the branch's rule in the same session (141.23 for 15 fixed nodes at 34): 5 %: 138.05 / 41.28; 2 sigma: 135.11 / 47.06;
4 sigma: 137.92 / 42.42. The two wide bands lose 11 % and 9 % at 28,946 tokens (41.28 / 46.61, 42.42 / 46.61) and none
reaches the fixed tree at 34 tokens, so none is in this change. The first request of each band (the warm-up) also pays
the width trials (121,022 tokens: 18.0 tok/s against 21.4 to 22.4 afterwards); the per-request table is in the receipt.

Tests: zig build test exited 0 on the code commit (the host tests drive width/measured directly: none calls XpuLanes.alone, so the
loop's wiring is covered by the device runs and the tables above only); new host tests (width follows the band's measured rounds; XPU_TREE caps; w-node tree
is the prefix of the 15-node one and tokens are equal at widths 1 to 15, greedy and seeded; lone pricing without
plain_guard). Device tests xpu-qwen27-tests 3 of 3 and the kernel contract (9 pass, 5 controls red) on the B70.

[ashhart]

Thanks @Mirrdhyn. This is what #2897 asked for: the lone tree's width now comes from lone_costs, and XPU_TREE stays a cap. The deep cells are a big win, 1.58 to 2.04 times main from 28,946 to 121,022 tokens, with one hash per prompt. The exactness argument holds: a tree of w nodes is the first w pops of the 15-node tree (grown and settle in zig/src/xpu/drafter_tree.zig), and the walk keeps the target's own pick. The receipt is thorough, and it helps that it says plainly where the branch falls short.

We also checked who else the core change reaches. lone_costs.enabled (zig/src/core/lanes/lone_costs.zig:15) still needs lone_cost_tile > 0 and a head plan. On main, only Qwen3.5 on CUDA sets a tile, and both of its drafters set plain_guard, so CUDA keeps its current path.

Before it can land

1. The 34-token cell. The branch is 3.6% below main there (136.14 against 141.23 tok/s). Short prompts are the common case, so we can't land a loss there. XPU (B70) Qwen3.8-27B: on code at 121,022 prompt tokens, drafted decode is slower than plain decode #2897 set the bar at or above the best fixed tree at each depth. Your own log points two ways:

   • The table prices 15 nodes at 43.0 ms from one trial, and 14 nodes at 39.8 ms from many rounds. A width priced from one timing, or from the warm-up's first rounds, shouldn't hold the stream off the cap.
   • A 14-node round on the branch takes 40.4 ms; a 15-node round on main takes 40.1 ms. So part of the gap may be the per-round sync (zig/src/native/xpu_lanes.zig:545) and the host pricing, not only the width. Please measure that overhead on its own.

   The first request in each band also pays the width trials (120 tok/s at 34 tokens). Every fresh server pays that, so please show it beside main's first request.

2. Rebase. The branch is 136 commits behind main. It conflicts in CHANGELOG.md and zig/src/native/draft_sources_xpu_test.zig, so CI hasn't run. It also conflicts with xpu: serve logprobs and top_logprobs on Qwen3.8-27B #3051, which we'd land first; its Note hook then moves into your walk. With xpu: a sampled Qwen3.8-27B row draws from a top-k kept on the card #2895 and xpu: serve logprobs and top_logprobs on Qwen3.8-27B #3051 in, your 24 lines would take xpu_lanes.zig past 600, and xpu: serve logprobs and top_logprobs on Qwen3.8-27B #3051 is moving its logprob code out to make room.

3. Sampled speed. The speed table is greedy only. Please add main against the branch at temperature 1.0, seeded, at 34 and 121,022 tokens. You already have these runs for the hashes.

4. Engine rounds and more streams. A single-stream engine round is now priced from the stream's own table with least_rows 0 (zig/src/core/lanes/engine.zig:234, zig/src/core/lanes/windows.zig:285), so it can pick no drafts. Please post main against the branch for one request the lone driver refuses, such as a structured-output or tool-call request, greedy and sampled. Please add one cell at --parallel 2 or wider. Both need equal hashes.

5. Two clocks, one table. The driver times the lattice, tree, verify and taps. The engine's round times flush, verify and the late draft. Both write the same keys and share Engine.lone_kept by band (xpu_lanes.zig:522, engine.zig:595). Please keep them apart, or measure the offset and show it doesn't change the choice.

6. One copy of the pricing. width() (zig/src/native/xpu_dflash2.zig:39-58) repeats the sequence at windows.zig:275-292: seed, prices, allocateDrafts, probe and trial. It also resets the core's round arena from the backend (xpu_dflash2.zig:47). Please move that sequence into one lone_costs function that both call, with least_rows as a parameter, so the two copies can't drift. As XPU (B70) Qwen3.8-27B: on code at 121,022 prompt tokens, drafted decode is slower than plain decode #2897 said, the core is the place to extend.

7. An existing test. zig/src/core/lanes/lone_costs_test.zig:105 adds s.rounds += 8 to a CUDA test. Please say why. If it makes the check stronger (past the deadline rather than before it), please confirm it also passes on main.

8. Changelog. Please end the line with (#3052, refs #2897), and update the 34-token figure once item 1 is fixed.

• Synthena (Ash's Agent)

[Mirrdhyn]

Measurements for items 1, 3 and 4, and where item 1 stands. Nothing is pushed yet. I don't have a fix for item 1 (details below), and the 34-token figure in the changelog waits on it. The rest does not: the rebase after #3051 (the Note hook into walk, the line budget, CI), the two clocks, one copy of the pricing in lone_costs, the explanation of s.rounds += 8, and the changelog suffix come with the next push.

All runs: one Arc Pro B70, this branch's parent 52c8f51e7 against its head d71c69445 (not rebased onto current main yet), the receipt's server flags (--parallel 1 --context 132096 --prompt-cache-gib 0), DFlash2, up to 256 new tokens. Every token_sha256 below is equal between the parent and the branch.

Item 1: the 34-token cell

Greedy, 5 fresh servers a side, 4 timed requests each after one warmup.

	tok/s, median of 20	ms a round
parent	141.51	40.04
branch, the rule	139.21 (−1.6 %; −2.1 % as the median of the 5 servers' medians, 138.60)	39.21
branch, width forced to 15	141.44	40.07
branch, forced to 15, no pricing, no bookkeeping, no sync	141.51	40.04

The receipt's −3.6 % (136.14 against 141.23) is the same card at 02e26dd51, two passes of three requests. The only change under zig/ since that commit is a comment, so I read 136.14 against 139.21 as protocol and run-to-run spread.

• Overhead. A probe around the sync at xpu_lanes.zig:545 records 329 µs, but that call is a second synchronize: settle already waits in verifyTree and commitPath (qwen27.zig:488, :521). Removing the sync, the pricing and the bookkeeping moves the round mean from 40.054 ms to 40.040 (σ 0.63, n = 860), so the probe's 329 µs is not time added to the round. width() takes 0.7 µs a round. Forced to 15 nodes with all of it left in, the branch is 141.44 tok/s against 141.51. So the loss is the widths the rule picks, not the sync or the host pricing.
• 14 against 15 nodes. Forced widths, mean round: 14 nodes, 39.63 ms on the branch (n = 880, σ 0.63) and 39.61 on the parent (XPU_TREE=14); 15 nodes on the parent, 40.05. The receipt's 40.4 ms was 14-node rounds chosen by the rule at 02e26dd51 (40.42, σ 1.37, n = 156).
• Prices. In this run's steady price table at 34 tokens, 15 nodes is 40.2 ms, close to its forced round (40.1). The widths from 9 to 14 are priced 3 to 6 ms above their forced rounds: 9 nodes at 41.2 against 37.8, 11 at 44.7 against 38.5, 12 at 44.9 against 38.9, 13 at 42.1 against 39.3, 14 at 43.1 against 39.7; 7 nodes is priced at 37.0, its forced round. On steady rounds the rule never picks 11, 12 or 14, picks 13 (the derived best) in 3 % of rounds and 9 in 5 %.
• First request of a fresh server: parent 128.21 tok/s, branch 120.40 (−6.1 %). Over the 5 servers' first requests (250 rounds) the rule tries 13 widths, 1 and 4 to 15.
• What the rule picks, and what a fixed tree would give. A tree of w nodes is the first w pops of the 15-node tree, so a round verified at 15 gives exactly what each narrower tree would have accepted in it. I logged that for every round at 34, 7,588, 28,946, 58,543 and 121,022 tokens (checked against rounds actually run at each forced width: identical tokens in all 2,104 matched rounds), and timed rounds at each forced width and depth. Round time is smooth, about 0.4 ms a node at 34 tokens and 21 ms a node at 121,022.
  • Derived tok/s (accepted tokens over round time, not the served rate; the served 15-node tree is 141.51): at 34 tokens, 11 to 15 nodes span 140.2 to 141.0, best 13 at 141.0, 15 at 140.5. At 7,588 the best fixed width is 6 (69.2); 15 is 59.6 and the rule 68.3. At 28,946 the best is 3 or 4; at 58,543 and 121,022 it is 3.
  • On steady rounds at 34 tokens the rule picks 15 nodes in 74 % of rounds and 7 in 16 %. Its expected tokens are not biased toward the narrow trees: on the rounds forced to 15 at 34 tokens (n = 132), expected over accepted runs from 0.95 to 0.99 across widths, 0.971 at 7 nodes and 0.992 at 15.
• Rules tried offline, on resampled rounds (this does not model how landing learns under each rule). Shrinking each round's expected tokens toward their running mean, with smoothed prices and a minimum of observations, reaches the 34-token plateau and stays above the best fixed tree at 28,946 and 58,543, but misses it by about 1 % at 7,588 and 121,022. One width per context band, from the measured acceptance, does not repair 34 tokens and gives up several tok/s at 28,946 and 58,543. None of the variants is at or above the best fixed tree at every depth.

So I don't have a rule that meets #2897's bar yet. Forcing 15 removes the served loss at 34 tokens, and 15 sits on that plateau (0.4 % under the derived best), but it is far from the best at 7,588, and nothing is measured between those two depths. Sampled, the simulation puts the rule above the cap at 34 tokens (161.7 against 159.2, best fixed 160.2 at 14 nodes) while the served item 3 cell is 0.991, so a cutoff would need its own sampled cells too. Would you review a cutoff in lone_costs (the cap below some context, the per-round rule above it, with measurements on both sides of the cutoff), or would you rather have a change to the per-round estimate, held to the best fixed tree at every depth?

Item 3: sampled

Temperature 1.0, top_k 20, top_p 0.95, seed 7. 5 fresh servers a side, alternating, one warmup and one timed request.

prompt tokens	parent	branch	ratio
34	150.32 (150.19 to 150.34)	148.94 (148.77 to 153.62)	0.991
121,022	10.10	19.09	1.89

Item 4: requests the lone driver refuses, and more streams

The lone driver refuses "draft": false, a JSON schema, tool_choice: "required" and a thinking budget (lane_host_lone.zig:18, the XPU flags at xpu.zig:166); an instrumented build logged no lone-driver round for them. Sixteen engine-round cells (those four, greedy and sampled with seed 7, a short prompt and about 8K tokens): every hash equal. Plain and JSON-schema greedy cells are at 1.000 to 1.003; tool and thinking cells run from 0.94 to 1.11 in both directions and JSON-schema sampled at about 8K is 0.970. These rates were not alternated, and I read the scatter as noise without having shown it. logprobs is refused with HTTP 400 on both sides, since this branch predates #3051.

total tok/s, prefill included	parent	branch
--parallel 2, two requests at once (34 and 7,588 tokens)	62.42	63.92
--parallel 4, four at once (34 to 30,966 tokens)	20.43	20.40

At both batch sizes each request's hash equals the parent's and equals the same request sent alone.

[ashhart]

Thanks @Mirrdhyn. The new runs isolate the overhead, add sampled and engine-round coverage, and keep the hashes equal. The short-prompt gate is still open. These items remain:

1. zig/src/native/xpu_dflash2.zig:39: fix the width choice under XPU (B70) Qwen3.8-27B: on code at 121,022 prompt tokens, drafted decode is slower than plain decode #2897's best-fixed-tree gate. At 34 tokens it still gives 139.21 versus 141.51 tok/s, and 120.40 versus 128.21 on the first request. A context cutoff needs agreement and measurements on both sides.
2. zig/src/native/xpu_lanes.zig:522 and zig/src/core/lanes/engine.zig:595: separate the two clocks' calibration tables, or measure their offset and show it leaves choices unchanged.
3. zig/src/native/xpu_dflash2.zig:39 and zig/src/core/lanes/windows.zig:284: move the shared selection sequence into lone_costs, taking least_rows; remove the backend's arena reset.
4. zig/src/native/draft_sources_xpu_test.zig:4: after xpu: serve logprobs and top_logprobs on Qwen3.8-27B #3051 lands, resolve the integration, carry its Note hook into walk, and keep the affected modules within 600 lines.
5. zig/src/core/lanes/lone_costs_test.zig:105: explain s.rounds += 8 and confirm the strengthened check passes on main.
6. zig/src/core/lanes/engine.zig:234: repeat the slower engine-only request cells with alternating main/head runs; the posted scatter doesn't yet establish noise.
7. zig/src/native/xpu_dflash2_lone_test.zig:171: run zig fmt; remove the trailing whitespace in docs/receipts/xpu-qwen27-lone-tree-width/exp/scatter-rule.patch:8 too.
8. CHANGELOG.md:7: update the short-prompt figure once fixed and end the line with (#3052, refs #2897).

• Synthena (Ash's Agent)