Skip to content

core lanes: a cut chance tail prices by the rule's rates, not as a measured zero (#2870) - #2979

Open
devopsfortunato wants to merge 1 commit into
ashhart:mainfrom
devopsfortunato:task/Tensorfold-Contribut+t-mwcrsks6-guta5r
Open

devopsfortunato wants to merge 1 commit into
ashhart:mainfrom
devopsfortunato:task/Tensorfold-Contribut+t-mwcrsks6-guta5r

Conversation

@devopsfortunato

@devopsfortunato devopsfortunato commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

Before, the core's shared draft rounds could not tell a cut draft tail from a measured zero chance. A family chain cut zeroes the cached tail of a stream's draft chances, and budgets priced those cached chances; a zero left by a cut therefore read as a real zero chance, so the round stopped paying for drafts that would have been accepted (a ratchet). After, Stream.draft_chances_cut marks the first cached chance a chain cut left unknown, and budgets prices only the measured prefix from the cache: the marked tail is priced by the rule's rates exactly as an uncached tail is. The marker follows the cache exactly — a family's probabilities hook marks it for the chances it just wrote, windows.allocate commits it with the cache, clears it with the cache, and leaves a retained cache's marker untouched. It changes only how many drafts a round asks for, so replies stay identical; with no marker set every path computes the same prices as before.

Checks on this head

  • Head commit: f1bc2c62f (base upstream/main eab305db0), one commit, three files, +47/−3.
  • The issue's required receipt, the new host test shared draft prices: a cut chance tail prices by the rule's rates, not as a measured zero, run both ways: with the marker ignored (main's rule, known = @min(cached.len, most)) it measures rows cut=2 grown=4 zero=2 and fails at expect(out[0] > out[2]) — the cut stream priced as measured zeros; with the change it measures rows cut=3 grown=3 zero=2 and both assertions hold (out[0] > out[2], out[1] == out[0]). The three streams hold the same cache {0.9, 0.4, 0, 0, 0}, three drafting heads share 11 rows against a 15-row demand, so the allocator has to choose a width.
  • zig build test-shared-lane-regressions-host --summary all → exit 0, 3/3 steps succeeded; 361/362 tests passed (1 skipped); the built binary run directly reports 361 passed; 1 skipped; 0 failed, including 116/362 shared_drafts.test.shared draft prices: a cut chance tail ... OK.
  • zig build test --test-timeout 10m → exit 0 on this head (f1bc2c62f), run on the contributor's aarch64 host; zig build test-golden → exit 0 (322/335 equal to server goldens, 0 unexpected differences).
  • CI on this head: the No GPU workflow run 43153606491 and the Zig format run 43153606519 are green — zig, zig-windows, zig-macos-dev-tools, fmt, pytest, release-paths, both benchmark-clients jobs succeeded; the two nightly-only jobs are skipped.
  • zig build test-cuda-host → exit 0 on retry; the first run failed only in cuda_disk_test ("expected 2, found 0"), which passed 2 of 3 direct runs on this host and is unrelated to these files.
  • zig fmt --check build.zig zig → clean. python3 tools/zig/lean_check.py → 6248 files, 345 problems, identical to the base on this worktree; no finding in the three touched files.

Receipt

Checklist

  • The title and description match the head commit.
  • The host test fails on main's rule and passes after the change.
  • Four-stream decode with DFlash2 is no slower than main (needs a GB10/M5 host; not run here).

@ashhart

ashhart commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Thanks @devopsfortunato for picking up #2870, and for writing down both of GLM's zero sources for whoever wires the first family.

The pricing rule is right. With no marker, known at zig/src/core/lanes/shared_drafts.zig:41 is the old @min(cached.len, most), so every path prices as before. With a marker, the cut tail continues from the last measured chance at the rule's rates. A cut at 0 drops to the uncached branch, as an empty cache does. We ran the new test both ways on f1bc2c62f: it fails with main's rule and passes with yours. CI is green, and the branch merges cleanly with today's main. No family sets the marker yet, and GLM's block path still stays off shared rounds (zig/src/families/glm/cuda_lanes.zig:103), so serving can't change here. We won't ask for the four-stream DFlash2 run on this PR; it belongs with the follow-up that wires a producer.

Three small things before it lands:

  1. A test for the allocate side. Only budgets is tested, with a marker set by hand. Removing the commit at zig/src/core/lanes/windows.zig:278 or the clear at :272 would fail no test. Could you add a host test that gives the fake backend a probabilities hook that writes chances and sets s.draft_chances_cut, runs allocate with allocate_shared_drafts on, and checks these three cases?

    • A head plan commits the marker with its cache. With draft_confidence set, the marker is capped at the prefix kept.
    • A plan with no chances (forced, or no drafts) keeps the previous cache and marker.
    • A copy plan clears both.

    Each check should fail when its line is removed.

  2. The error path. windows.zig:251 clears the marker before planChances, and :267 puts it back only when that call returns. If the hook fails (or NoDraftProbabilities is raised), the stream keeps its old chances without their marker, and that's the zero-as-measured reading this PR removes. An errdefer p.stream.draft_chances_cut = prior; after :251 keeps the marker with its cache on that path too.

  3. Say it where a family author will look. The only place a producer can set the marker is inside its probabilities hook. Could the doc comment on that field (zig/src/core/lanes/backend.zig:121) say that the hook sets s.draft_chances_cut to the first chance it could not measure? That covers both a cut chain and chances past what it holds. Keep it to one line, per the repo's comment rule.

Once these are in and CI is green, it can go on the landing queue.

  • Synthena (Ash's Agent)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

core lanes: shared rounds can't tell a cut draft tail from a measured zero chance

2 participants