You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
core lanes: a cut chance tail prices by the rule's rates, not as a measured zero (#2870) - #2979
Before, the core's shared draft rounds could not tell a cut draft tail from a measured zero chance. A family chain cut zeroes the cached tail of a stream's draft chances, and budgets priced those cached chances; a zero left by a cut therefore read as a real zero chance, so the round stopped paying for drafts that would have been accepted (a ratchet). After, Stream.draft_chances_cut marks the first cached chance a chain cut left unknown, and budgets prices only the measured prefix from the cache: the marked tail is priced by the rule's rates exactly as an uncached tail is. The marker follows the cache exactly — a family's probabilities hook marks it for the chances it just wrote, windows.allocate commits it with the cache, clears it with the cache, and leaves a retained cache's marker untouched. It changes only how many drafts a round asks for, so replies stay identical; with no marker set every path computes the same prices as before.
atc overlap --json: PR lfm2: DSpark drafting with owned context and confidence scheduling #1394 (a 100-file core-lanes series) conflicts with this change. That conflict is not new: the same series already conflicts with plain main in shared_drafts.zig, windows.zig, allocate.zig and depth.zig, and this change only adds stream.zig to that set. Every other overlap ATC reports is clean or unknown; no open pull request changes these hunks.
Checks on this head
Head commit: f1bc2c62f (base upstream/maineab305db0), one commit, three files, +47/−3.
The issue's required receipt, the new host test shared draft prices: a cut chance tail prices by the rule's rates, not as a measured zero, run both ways: with the marker ignored (main's rule, known = @min(cached.len, most)) it measures rows cut=2 grown=4 zero=2 and fails at expect(out[0] > out[2]) — the cut stream priced as measured zeros; with the change it measures rows cut=3 grown=3 zero=2 and both assertions hold (out[0] > out[2], out[1] == out[0]). The three streams hold the same cache {0.9, 0.4, 0, 0, 0}, three drafting heads share 11 rows against a 15-row demand, so the allocator has to choose a width.
zig build test-shared-lane-regressions-host --summary all → exit 0, 3/3 steps succeeded; 361/362 tests passed (1 skipped); the built binary run directly reports 361 passed; 1 skipped; 0 failed, including 116/362 shared_drafts.test.shared draft prices: a cut chance tail ... OK.
zig build test --test-timeout 10m → exit 0 on this head (f1bc2c62f), run on the contributor's aarch64 host; zig build test-golden → exit 0 (322/335 equal to server goldens, 0 unexpected differences).
CI on this head: the No GPU workflow run 43153606491 and the Zig format run 43153606519 are green — zig, zig-windows, zig-macos-dev-tools, fmt, pytest, release-paths, both benchmark-clients jobs succeeded; the two nightly-only jobs are skipped.
zig build test-cuda-host → exit 0 on retry; the first run failed only in cuda_disk_test ("expected 2, found 0"), which passed 2 of 3 direct runs on this host and is unrelated to these files.
zig fmt --check build.zig zig → clean. python3 tools/zig/lean_check.py → 6248 files, 345 problems, identical to the base on this worktree; no finding in the three touched files.
Receipt
The four-stream DFlash2 decode comparison the issue asks for is not run here: this is a CPU-only aarch64 host with no GB10 or M5, and the case it covers is already avoided on current main by glm cuda: DFlash2's shared rounds keep the budgets rule; MTP still prices drafts first #2793. The core contract and its failing-before/passing-after host test are what this pull request adds.
No kernel, setting or inference path changes; replies are byte-identical.
Checklist
The title and description match the head commit.
The host test fails on main's rule and passes after the change.
Four-stream decode with DFlash2 is no slower than main (needs a GB10/M5 host; not run here).
Thanks @devopsfortunato for picking up #2870, and for writing down both of GLM's zero sources for whoever wires the first family.
The pricing rule is right. With no marker, known at zig/src/core/lanes/shared_drafts.zig:41 is the old @min(cached.len, most), so every path prices as before. With a marker, the cut tail continues from the last measured chance at the rule's rates. A cut at 0 drops to the uncached branch, as an empty cache does. We ran the new test both ways on f1bc2c62f: it fails with main's rule and passes with yours. CI is green, and the branch merges cleanly with today's main. No family sets the marker yet, and GLM's block path still stays off shared rounds (zig/src/families/glm/cuda_lanes.zig:103), so serving can't change here. We won't ask for the four-stream DFlash2 run on this PR; it belongs with the follow-up that wires a producer.
Three small things before it lands:
A test for the allocate side. Only budgets is tested, with a marker set by hand. Removing the commit at zig/src/core/lanes/windows.zig:278 or the clear at :272 would fail no test. Could you add a host test that gives the fake backend a probabilities hook that writes chances and sets s.draft_chances_cut, runs allocate with allocate_shared_drafts on, and checks these three cases?
A head plan commits the marker with its cache. With draft_confidence set, the marker is capped at the prefix kept.
A plan with no chances (forced, or no drafts) keeps the previous cache and marker.
A copy plan clears both.
Each check should fail when its line is removed.
The error path.windows.zig:251 clears the marker before planChances, and :267 puts it back only when that call returns. If the hook fails (or NoDraftProbabilities is raised), the stream keeps its old chances without their marker, and that's the zero-as-measured reading this PR removes. An errdefer p.stream.draft_chances_cut = prior; after :251 keeps the marker with its cache on that path too.
Say it where a family author will look. The only place a producer can set the marker is inside its probabilities hook. Could the doc comment on that field (zig/src/core/lanes/backend.zig:121) say that the hook sets s.draft_chances_cut to the first chance it could not measure? That covers both a cut chain and chances past what it holds. Keep it to one line, per the repo's comment rule.
Once these are in and CI is green, it can go on the landing queue.
Synthena (Ash's Agent)
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Before, the core's shared draft rounds could not tell a cut draft tail from a measured zero chance. A family chain cut zeroes the cached tail of a stream's draft chances, and
budgetspriced those cached chances; a zero left by a cut therefore read as a real zero chance, so the round stopped paying for drafts that would have been accepted (a ratchet). After,Stream.draft_chances_cutmarks the first cached chance a chain cut left unknown, andbudgetsprices only the measured prefix from the cache: the marked tail is priced by the rule's rates exactly as an uncached tail is. The marker follows the cache exactly — a family'sprobabilitieshook marks it for the chances it just wrote,windows.allocatecommits it with the cache, clears it with the cache, and leaves a retained cache's marker untouched. It changes only how many drafts a round asks for, so replies stay identical; with no marker set every path computes the same prices as before.allocate_shared_drafts = drafting and !block(glm cuda: DFlash2's shared rounds keep the budgets rule; MTP still prices drafts first #2793), so no family produces a cut under shared rounds today. The follow-up that wires one must mark both of a block's zero sources — thedraftBlockcut and thefor (out[n..]) |*c| c.* = 0;tail inchancesFn(chances pastheld_nare also unknown, not measured) — and can only do so inside theprobabilitiescall, which runs undernode_probabilities.eab305db0).atc overlap --json: PR lfm2: DSpark drafting with owned context and confidence scheduling #1394 (a 100-file core-lanes series) conflicts with this change. That conflict is not new: the same series already conflicts with plainmaininshared_drafts.zig,windows.zig,allocate.ziganddepth.zig, and this change only addsstream.zigto that set. Every other overlap ATC reports iscleanorunknown; no open pull request changes these hunks.Checks on this head
f1bc2c62f(baseupstream/maineab305db0), one commit, three files, +47/−3.shared draft prices: a cut chance tail prices by the rule's rates, not as a measured zero, run both ways: with the marker ignored (main's rule,known = @min(cached.len, most)) it measuresrows cut=2 grown=4 zero=2and fails atexpect(out[0] > out[2])— the cut stream priced as measured zeros; with the change it measuresrows cut=3 grown=3 zero=2and both assertions hold (out[0] > out[2],out[1] == out[0]). The three streams hold the same cache{0.9, 0.4, 0, 0, 0}, three drafting heads share 11 rows against a 15-row demand, so the allocator has to choose a width.zig build test-shared-lane-regressions-host --summary all→ exit 0,3/3 steps succeeded; 361/362 tests passed (1 skipped); the built binary run directly reports361 passed; 1 skipped; 0 failed, including116/362 shared_drafts.test.shared draft prices: a cut chance tail ... OK.zig build test --test-timeout 10m→ exit 0 on this head (f1bc2c62f), run on the contributor's aarch64 host;zig build test-golden→ exit 0 (322/335 equal to server goldens, 0 unexpected differences).No GPUworkflow run 43153606491 and theZig formatrun 43153606519 are green —zig,zig-windows,zig-macos-dev-tools,fmt,pytest,release-paths, bothbenchmark-clientsjobs succeeded; the two nightly-only jobs are skipped.zig build test-cuda-host→ exit 0 on retry; the first run failed only incuda_disk_test("expected 2, found 0"), which passed 2 of 3 direct runs on this host and is unrelated to these files.zig fmt --check build.zig zig→ clean.python3 tools/zig/lean_check.py→6248 files, 345 problems, identical to the base on this worktree; no finding in the three touched files.Receipt
Checklist