Repository navigation
Conversation
wojo
marked this pull request as draft
November 7, 2026 03:34
Owner
|
Thanks @wojo. We read what this adds over #1411. With
|
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 7, 2026
TF_GLM_KV_SPLIT is off by default; in this part every rank still holds every latent row, and a class holds a whole window, so a split server can admit fewer streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7 of 8 tasks
4 tasks done
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 7, 2026
TF_GLM_KV_SPLIT is off by default; in this part every rank still holds every latent row, and a class holds a whole window, so a split server can admit fewer streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Thanks @ashhart. Restacked on #1411's new head (
Engine checks on Host on |
This was referenced Nov 8, 2026
Owner
|
Thanks @wojo. All five points are in: layout 21 with the shapes in between named (
Also, #2528 (the reclaim chain) is ready to land and changes |
Contributor
Author
|
Thanks @ashhart. Rebased onto main
One behaviour change to flag: New test in the first commit: Hash rows on
All 1028 pages on each build; set adds Host: |
8 tasks done
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 8, 2026
…t one rank, without pages or with host pages The switch for ashhart#1431's per-rank latent book: unset, 0 or off (the default) and 1 or on; any other value is refused naming the choices. Every rank resolves it inside the watched start, after TF_KV_PAGES: one rank is KvSplitNeedsRanks (checked first, since auto pages are off there), pages off or fp8 windows KvSplitNeedsPages, TF_GLM_KV_HOST KvSplitOnHost, and TF_KV_SHARE KvSplitShare (the store's pages come from any owner's class). The fingerprint carries it (layout 21), so ranks started with different values stop with kv_split named. Nothing books by owner yet: this commit only reads, agrees and refuses. Its rows: environment.zig checks the same four words when the CUDA backend starts (docs/environment.md), and profile.zig registers it beside TF_GLM_KV_HOST for glm5_next, identical, 1 and on reported as set, evidence ashhart#2415, since it changes how pages are booked and how many streams fit (docs/profiles.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 8, 2026
TF_GLM_KV_SPLIT is off by default; in this part every rank still holds every latent row, and a class holds a whole window, so a split server can admit fewer streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Follow-up: main moved under this right after the push above (#2444 C2 in |
4 tasks done
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 9, 2026
…t one rank, without pages or with host pages The switch for ashhart#1431's per-rank latent book: unset, 0 or off (the default) and 1 or on; any other value is refused naming the choices. Every rank resolves it inside the watched start, after TF_KV_PAGES: one rank is KvSplitNeedsRanks (checked first, since auto pages are off there), pages off or fp8 windows KvSplitNeedsPages, TF_GLM_KV_HOST KvSplitOnHost, and TF_KV_SHARE KvSplitShare (the store's pages come from any owner's class). The fingerprint carries it (layout 21), so ranks started with different values stop with kv_split named. Nothing books by owner yet: this commit only reads, agrees and refuses. Its rows: environment.zig checks the same four words when the CUDA backend starts (docs/environment.md), and profile.zig registers it beside TF_GLM_KV_HOST for glm5_next, identical, 1 and on reported as set, evidence ashhart#2415, since it changes how pages are booked and how many streams fit (docs/profiles.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
|
Thanks @wojo for the restack onto main
CI is green on this head, and GitHub shows it mergeable. On the CHANGELOG line: you don't need to push again for it. It sits alone in the head commit, so we'll leave that commit out when this lands and write our line from your wording. Your device rows were on
|
8 tasks done
…re each byte sits in its lease The rows store's view of a GLM kept state, without the store: the words (format, position, head rows, ring bases, KDA copy bits), the KDA prefix, each DSA layer's latent rows, scales and whole pools by position, its rings and pools in progress as the state's own, the head's caches at a whole verify window (rows past the head's own read as zeros), then the tail. Copies run a page of the lease at a time. Nothing is wired to --learn-conversations yet.
…state on disk Rank 0's engine thread leases, fills, reads and seals each rank's copy through the follow link (cuda_disk_ranks: blank, put, take, seal, prefixes). The store's image is rank 0's image followed by each other rank's prefix, the one part a rank holds alone; the rest are the same bytes on every rank, checked at start by the identity probe. A blank state is refused by restore until its seal. A blank whose disk step fails on the link gives rank 0's lease back, and its id goes on the dropped list, so the next step drops it on any rank the step reached; the id is not used again. An NCCL failure there already ends every rank (the liveness lines), but a CUDA error in the step's launch or upload returns to the disk read, which tries one shorter state and then prefills while the server goes on. cuda_disk_ranks_test.zig runs the steps on host memory with a fake link: that failed blank, a blank state refusing bytes past its words and front until sealed (DiskOrder), each BadStep a follower refuses before its states or the link, and each other rank's shared bytes taken past its prefix and compared to rank 0's (RanksDiffer, in ranks.same, which the probe calls). attach's refusals of the shared store and of more ranks than prefix_buf holds (KvShareWithLearnConversations, TooManyRanks) have their own test in cuda_glm_disk.zig. link and prefix_buf stay process-wide: the process serves one model. With TF_GLM_KV_LIVE_HALF=1 a keep holds one KDA half and the disk image both, so GLM refuses --learn-conversations with it by name at startup (cuda_refusal.zig); the test that pinned GLM refusing --learn-conversations outright now pins that it takes the flag alone and refuses it beside live-half keeps. Rank 0's link becomes the disk steps' link in `follow.Link.lead` (which also sets whether the other ranks begin asleep), at TP only, and `Link.deinit` clears it, so cuda_native.zig stays at its 600 lines. cuda_disk_ranks_test.zig: one rank leaves the steps local, two set the link, another link's deinit leaves it, its own clears it; the test fails with the clear removed.
…sk state into that storage A family whose kept states live in preallocated storage it reuses (GLM's snapshot pages on CUDA) and cannot trim never had a disk read chosen: the selection counted its spare storage as fixed and its kept states as not reclaimable, so no state ever fit. Such a family's room for a read is now its spare storage plus what evicting unborrowed states frees, and a read is ready once the family says its spare storage takes the state, as a keep's admission already asks. Families with trim, or without reused storage, are unchanged.
…t, as before The disk read's early admission for a family whose spare storage takes the state now applies only to a family without trim, as the selection's capacity already does. A family with trim, such as Flash Next's or Qwen3.8-27B's pools on Metal, makes room for a read by trimming its spare storage and evicting, exactly as it did before, and a new test pins that its read stays within the budget when its reuses answers yes.
…he state's rows An index ring based at `base` holds the rows from `base` up to the state's position, and the image keeps at most a ring's rows of each. The seal now checks the words' main base against the state's position and the head's base against its rows, so an image whose words name a ring that starts past them, or too far back to be held, is refused before the blank state is made whole.
…/O worker GLM's store uses The write-behind gave GLM's store its own I/O worker, while the grid test wrote on the test's thread (sync_writes). It now runs on each, drained by write.drain, so a worker path that loses the grid's chunk starts fails the test.
…rn-conversations alone keeps its states Since ashhart#1413 CUDA serves --spill-gib and --checkpoint-slots, and --learn-conversations admits GLM, so --learn-conversations --spill-gib (or --checkpoint-slots) opened GLM and ignored the flag: its disk store writes every rank's part between passes and stages nothing at eviction. GLM now refuses both by name. Its store's spill stays off, since an eviction would gather on rank 0 mid-pass while the other ranks wait in the gate's all-reduce. docs/api.md lists every CUDA family that keeps states on disk.
…he shared-store refusal names TF_KV_SHARE - A kept state's device pages are read out with cuda.memory.download, which stages through the pinned buffer on an integrated GPU (ashhart#2103, ashhart#2289), as every other synchronous readback does; the copy back in stays a direct HtoD. - The refusal beside share_pages names TF_KV_SHARE, the switch behind it since ashhart#2211, in its message and comment. - The disk image test's driver answers cuCtxGetDevice and cuDeviceGetAttribute (a discrete card).
…e owner, and a read refuses another by name The image's words gain three: the ranks that wrote it, its layout and the rank that owns its latent rows. Every image is unsplit so far (every rank holds every row): it records this server's world, layout 0 and no owner. A read refuses words of another world (DiskWorld) or another layout, or an owner its layout does not have (DiskLayout), before the blank state is sealed. The words are 72 bytes, the seal step 18 words. ashhart#1431's split states fill the layout and owner without another format. A disk read forgets a state's file refused as DiskWorld or DiskLayout after one recall, as it does one refused as DiskWords (rows_read.refused); rows_refusal_test.zig's cases include both.
… attach, so a long state is staged whole cuda_glm_disk.attach opened the store with rows.Rows.open, which keeps the 1 GiB default staging permit and skips the floor rows.attach applies (the longest state's charge, within the RAM store's budget). At about 18.5 KiB a token that stopped every state past roughly 55K tokens: `imprint: saving 124928 tokens skipped (StagingFull)`, whatever --learn-gib said, and a long conversation was recomputed cold after a restart (found by crystal8194 on ashhart#1411). attach now goes through rows.attach, as Qwen 3.5 27B does, with `longest` the capacity's whole pools (`image.longest`, the last position `parts` takes; rows.attach falls back to the default when pricing it fails, so the position matters). Spill stays off and the slots unset, as before. The start-up line gains the permit: "a save stages up to N MiB". Tests: - test-cuda-disk-open-host: GLM's own open over a store of 18,944-byte tokens. A 124,928-token state's charge is past the 1 GiB default and within the permit, which is the longest state's charge; with a RAM store too small to keep the state the permit stays the default. It fails with the floor gone (the permit is 1 GiB). It prices the permit a save's stage step asks; it does not write 2.3 GiB. - The GLM set: `longest` is a position the store's parts take at capacities on and off a pool's edge, and the next pool past it is refused.
…id from one class, and one class is today's pool A pool can hold W owner classes (Pool.initClasses, Layout.classes): page ids c, c + W, c + 2W ... form class c, each on its own free stack, as DeepSeek's sessions pool books ids by residue (pg = local*W + r). A lease names its class (Lease.owner) and every id it takes, adopts or shares is of that class; a promise fails when its class lacks the pages, though another has them. The owner rule lives in one function: roomiest, the class with the most available pages, the lowest on ties (ashhart#1313's). fitsAny answers as a promise from it would. planClasses floors a plan's pages to a multiple of the classes and refuses a class that holds less than one stream's window (ClassTooSmall); plan's error set is unchanged, and at one class planClasses is plan for the same budget, the cache's slots and slots_only (ashhart#2358) included. The class book lives in kv_pool_classes.zig (planClasses, the per-class counts, roomiest, fitsAny and the stacks' push and pop, and the class tests' imports); kv_pool.zig keeps the fields and names them, and promise and hold keep main's reclaim walk (ashhart#2528's chain) as it is: the class test comes after it. Above one class, runs are refused (claimRun null, coverRun as cover), since a run spans every class, and so is a reclaim store: joinReclaim returns OwnerClasses, since a store's eviction frees pages of any class and a promise needs its own class's. Nothing joins one there today (GLM refuses TF_KV_SHARE with the split); the refusal keeps it so. One class is today's pool, id for id and in the same order: init is initClasses(..., 1), and a scripted run of promise, cover, trim, finish, hold and release gives the trace main records (pinned by its hash). kv_pool_trace_test.zig goes past that script on one class: 400 seeded steps on 32 pages through adopt, share, drop, runs, reclaims and a shuffled stack, every operation taking effect, its store joining as any store does (joinReclaim), its hash recorded on main's kv_pool.zig (ddd2df8, again on 9c9bf54, and on f03e5aa, after ashhart#2528's chain).
…t one rank, without pages or with host pages The switch for ashhart#1431's per-rank latent book: unset, 0 or off (the default) and 1 or on; any other value is refused naming the choices. Every rank resolves it inside the watched start, after TF_KV_PAGES: one rank is KvSplitNeedsRanks (checked first, since auto pages are off there), pages off or fp8 windows KvSplitNeedsPages, TF_GLM_KV_HOST KvSplitOnHost, and TF_KV_SHARE KvSplitShare (the store's pages come from any owner's class). The fingerprint carries it (layout 21), so ranks started with different values stop with kv_split named. Nothing books by owner yet: this commit only reads, agrees and refuses. Its rows: environment.zig checks the same four words as a glm row, when GLM's CUDA family opens on every rank, so a typo never stops another family (docs/environment.md; cuda_env_test.zig), and profile.zig registers it beside TF_GLM_KV_HOST for glm5_next, identical, 1 and on reported as set, evidence ashhart#2415, since it changes how pages are booked and how many streams fit (docs/profiles.md).
…s lie in two regions, each with its own table The pool's arena is the same bytes, laid out as two regions: every page's pooled keys first, a pooled part apart, then every page's latent rows, a latent part apart. Both strides come from the weights (the DSA layers and the head times a cache's part), so a thin model's pages are laid out alike. Each paged sequence gets a second table after its first (State.ltable), and Caches a second (pkt): the latent launches (lat_write, the dense and sparse chunks, and dsa_tokens' rows) take the latent table and stride, the pooled-key launches (pool_keys, scores) the pooled table and stride. No kernel or launch is new. attend refuses a pooled table without its own stride before it launches. Every rank still holds every row, so the latent table holds the same ids: TF_KV_POISON fills both parts of a page, and kept states copy the same bytes. Off, every launch argument, table write and fill is as before.
…wner's class, sent with begin and checked by every rank With the switch on, the pool has an owner class a rank (Layout.classes = TF_GLM_WORLD). Rank 0 picks a stream's owner when it opens, by the pool's rule (the class with the most available pages, the lowest on ties), and takes every page from that class; admission asks whether that class holds the request. begin carries the owner in its unused depth word, and every other rank checks each id is of it before it maps anything (ResidueMismatch, or BadStep for an owner past the classes). The pool step carries the switch, and a rank whose own differs refuses it (RankMismatch). Every rank refuses a pool whose class holds less than one stream's window (ClassTooSmall), and the plan floors the pages to a multiple of the classes and refuses such a class with its numbers; it plans from the budget main's plan builds, the cache's `slots` and slots_only (ashhart#2358) included. A request longer than a class is refused by name, never requeued. The lane cost cache keys its prices on the switch (keyShape's split word): under it the timing stream books its pages from a class. Every rank still holds every latent row, so the owner is bookkeeping only for now; the start-up line says so, and the idle line counts each class. Off, there is one class: ids, admission, begin and the pool step are as before.
…r classes, each prompt on one, the classes in turn The check resolves the switch as serving does (cuda_start.splitting, refused by name at one rank) and, with it on, makes an owner class a rank of two windows and a page over each. Each prompt's paged stream and the stream that shuffles its pages open on the same class, k % W for prompt k, so every class serves and its pages still come out of order (jumps counted by W). Every stream is compared with its twin on a whole window, byte for byte, and every class must be idle after. Off, the check is as before.
…th regions of every leased page, once A paged sequence on two regions (every page's pooled keys first, then every page's latent rows) and one on interleaved pages, both on ten ids out of order, at a length whose latent rows and pooled keys reach every page, with and without the MTP head. The split plan's paged pieces, counted on a map of the pages, read every leased page in both regions, each row's bytes exactly once, and nothing else (no overlap, none missed); the rows are placed from the geometry alone, not from the state's fields. With the same rows in both layouts (every row on every rank), the split plan's bytes, in its order, equal the interleaved plan's and so do the lengths. Both tests fail when the pooled parts step by the latent stride (x.ps) or the latent parts by the pooled one. glm_snapshot_host.zig is at 600 lines, so the tests have their own file in the same step.
…links, Header bytes and all, to a follower's loop Rank 0 is a Leader on the pages' fake engine with a Link on host memory: Leader.pool and Leader.beginOn encode through Link.send's put path (tf_glm_put_words run on the host, word for word) and a host NCCL that keeps each broadcast. The pool step's 64 bytes carry the switch in flags, and begin's carry the lease's owner in depth, with its ids after; off, both are byte for byte the headers main sends (flags 0, depth 0). A second rank decodes them in follow itself, through Link.receive: its pool books an owner class a rank, and slot 1 maps rank 0's ids under that owner. A begin whose owner on the wire is not its ids' class is refused (ResidueMismatch) with nothing mapped, and a follower started without the switch refuses the pool step (RankMismatch) and makes no pool. Mutants, each alone: begin's depth left 0, the pool's flag dropped, the follower's begin calling map (owner 0) and its pool step ignoring the flag; each fails at least one of the three tests. The pages' split test's Fake is made pub for it.
…ions, each with a control given the other table No launch is new, so glm_cuda_records.zig is unchanged; what the split changes is the geometry the eight paged launches meet. contract_glm_pages.zig gains Geometry: in an arena of n pages, every page's pooled keys first (2 x 32,768 bytes apart), then every page's latent rows (2 x 524,288 apart), the second cache tested. With regions on, tf_glm_lat_write and tf_glm_chunks_16/32 (dense and sparse) take the latent region's table and stride, tf_glm_dsa_tokens rows of that stride (1,024 a page, not 1,088), and tf_glm_pool_keys and tf_glm_scores the pooled region's. Off, every argument and byte is as before. contract_glm_regions.zig's Set holds the eight cases at either geometry; contract_tests.zig registers the eight on two regions (2 lines), and Controls now holds both sets as controls (8 -> 16). A region control gives each launch the other region's table, a second shuffle of the same pages that differs at every entry (sparse chunks: token rows read through it), so its rows land on or come from other pages. glm-index-check gains a paged case at the pooled stride (12 x 32,768, which tf_glm_index_scores' fast path takes) with a latent table of its own for the token lists. docs/kernel-contract.md says so. Host tests (test-glm-page-contract-host, 3 -> 5): sixteen controls, the last eight on two regions; a disabled control in either set is refused; the region cases register apart from the interleaved ones at the same served shapes as their controls; the regions fill the interleaved arena exactly, pooled keys first, and off places everything as before; a region control's table differs at every entry (1-11 pages, 64 seeds each). Mutants, each caught: the fix-up of equal entries dropped, the latent rows placed at the first cache, Set ignoring regions, the region controls not planted. The GPU gates (each case green, each control red) are not run here; the contract's body cross-builds for aarch64-linux with empty images (-Dfatbins, -Dcontract-controls), which is what analyzes it.
TF_GLM_KV_SPLIT is off by default; in this part every rank still holds every latent row, and a class holds a whole window, so a split server can admit fewer streams.
Contributor
Author
|
Rebased onto main |
This was referenced Nov 13, 2026
8 tasks done
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 13, 2026
…t one rank, without pages or with host pages The switch for ashhart#1431's per-rank latent book: unset, 0 or off (the default) and 1 or on; any other value is refused naming the choices. Every rank resolves it inside the watched start, after TF_KV_PAGES: one rank is KvSplitNeedsRanks (checked first, since auto pages are off there), pages off or fp8 windows KvSplitNeedsPages, TF_GLM_KV_HOST KvSplitOnHost, and TF_KV_SHARE KvSplitShare (the store's pages come from any owner's class). The fingerprint carries it (layout 21), so ranks started with different values stop with kv_split named. Nothing books by owner yet: this commit only reads, agrees and refuses. Its rows: environment.zig checks the same four words as a glm row, when GLM's CUDA family opens on every rank, so a typo never stops another family (docs/environment.md; cuda_env_test.zig), and profile.zig registers it beside TF_GLM_KV_HOST for glm5_next, identical, 1 and on reported as set, evidence ashhart#2415, since it changes how pages are booked and how many streams fit (docs/profiles.md).
wojo
added a commit
to wojo/TensorFold
that referenced
this pull request
Nov 13, 2026
TF_GLM_KV_SPLIT is off by default; in this part every rank still holds every latent row, and a class holds a whole window, so a split server can admit fewer streams.
Owner
|
Thanks @wojo. The Nothing further is needed from you. This still follows #1411, then our two-GB10 main/unset/set checks with
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current head (2026-11-12, evening):
af17f6c8fon main223da7659Rebased onto main
223da7659; CHANGELOG-only conflicts, no other change to the commits. No device rows on this exact head: part 4's build of 12 Nov (which contains these commits, one main earlier) gave the same token hashes as main with the setting off and on.Host on this head:
fmt, lean 341 vs 341,pr_check0 errors,test-cuda-host6,469 of 6,507 with 38 skipped (fresh cache), aarch64nativeandcompile-cuda-tests.Part 1 of #1431 (the latent split across ranks), as agreed there on 10-30. Stacked on #1411: please review the last 9 commits (
e57ae2ab4..6398a9627); the 10 below them are #1411's commits (its headd730aff64, here as7ec0621d2on main6c5cef894). All 19 are authored and committed with my noreply address, no co-author trailers. The disk image header'sworld,layout,ownermoved down into #1411.With
TF_GLM_KV_SPLIT=1the book, the owner, the wire and both tables are on the GPUs, and every rank still holds every latent row, so it saves no memory (the start-up line says so). Off, today's bytes, pinned by tests.Commits
e57ae2ab4Owner classes in corekv_poolon DeepSeek's residue rule: one arena, one id space, a free stack per class (pagec, c+W, c+2W…is classc),Lease.owner, the owner rule in one function (roomiest: most available pages, lowest rank on ties),fitsAny,planClasseswithClassTooSmall. One class is today's pool id for id: a pinned 40-step run replays the trace recorded on main (738 entries, hash0x6a51e5e6ca5c61f6). Six mutants, all caught.3c628d433TF_GLM_KV_SPLIT(.scope = .glm: checked when GLM's CUDA family opens, like the otherTF_GLM_*rows), agreed by every rank (fingerprint layout 22; 19 is glm cuda: a prompt step takes two chunks while no other stream decodes, each one's exchanges under the other's sites #1423's, 20 glm:TF_GLM_KV_LIVE_HALF=1keeps the live half of the KDA state alone in a kept prompt state #2181's, 21 cuda/roce: the one-shot RoCE all-gather at any world (mesh), and the DFlash2 drafter MLP split across every rank #2205'sroce_wide), refused by name: one rank, no pages,TF_KV_SHARE(KvSplitShare, fromcuda_start.sharing, with a host test that fails without it),TF_GLM_KV_HOST(KvSplitOnHost).2b3ddf221Two regions, two tables: pooled keys first, then latent rows;pt/psstay the latent table and one pooled-key table (Caches.pkt) is added. No launch or kernel is new.7ad3ba8f5Owner on the wire:planpasses the layout tobudgetOfbeforeplanClasses; the lane cost key (keyShape) includeskv_split, twelfth case. rank 0 picks it,begincarries it, every rank checks each id's class (ResidueMismatch). A class shorter than a window is refused at start; a request longer than a class is dropped by name. Start-up and idle lines per class.f39226208glm-paged-checkruns on owner classes when the switch is on.e069c67ffHost: a kept state's snapshot plan reads both regions of every leased page, once.62d2bc6f6Host: the pool andbeginsteps cross two actual links, Header bytes and all, to a follower's loop.1635d1e3dKernel contract: GLM's eight paged cases on two regions, each with a control given the other table.6398a9627CHANGELOG under Unreleased.Host (12 Nov, this head
6398a9627):fmt, lean 345 vs 345,pr_check0 errors,test-cuda-host6,432 of 6,470 with 38 skipped (fresh cache), aarch64nativeandcompile-cuda-tests, the 14 named host steps.contract_tests.zigis 600 lines (the regions set is created and freed incontract_glm_controls.zig);cuda_native.zig587.Host (10 Nov, head
7fcbcebf8): 15 of 16 suites rc 0, lean 0 new problems,fmtclean,kv_pool_trace_testhash unchanged,aarch64-linux-gnunative build at every commit.test-glm-startup-hostadds theKvSplitSharecase.test-cuda-host: 5,888 of 5,926 pass; its 2 are main'simages_generations_test"past 16 KiB" cases, which hang on macOS on6e9af9302and are fixed on main since75b2eaffd/783fa6324.On GPUs: two GB10s at TP=2
On
e0990375d(2026-11-12), the same 19 commits on main3885dceb2. The head above is that stack rebased over main's3885dceb2..6c5cef894(dots.mocr on CUDA; it touches none of this PR's files) and has host checks only. Unset and set (TF_GLM_KV_SPLIT=1 TF_KV_POISON=1) give the same 15 token hashes as main3885dceb2and as #1411's head, all four built and run in one window: 3 decodes, 3 cold 18.7K prompts, 1 and 4 concurrent, a 30K conversation's turns 1 and 2 (turn 2 now quotes three marker sentences planted in the conversation), drafted and"draft": false. Decode 33.1-33.9 / 32.8-33.5 / 33.4-33.9 tok/s, cold first token 14.99-15.02 / 14.99-15.01 / 14.96-14.99 s, 4 concurrent 47.69 / 47.75 / 47.51 tok/s (main / unset / set).glm-paged-checkunset and set: PASS (set: 2 classes of 36 pages, idle after).glm-index-check: PASS. Kernel contractregions: 8 kernels pass gates 1-4, 8 controls red as planted. Rank 1 logged no errors. At--context 1048576 --parallel 8the set server refuses by name (one stream takes 2,049 pages, a class has 1,248).On
7fcbcebf8(2026-11-10): unset and set (TF_GLM_KV_SPLIT=1 TF_KV_POISON=1) give the same 15 token hashes as main6e9af9302(main's from the same day's run): 3 decodes, 3 cold 18.7k prompts, 1 and 4 concurrent, a 30k conversation's turns 1 and 2, drafted and"draft": false.glm-paged-checkunset and set: PASS (set: 2 classes of 36 pages, idle after).glm-index-check: PASS. Kernel contractregions: 8 kernels pass gates 1-4, 8 controls red as planted. Rank 1 logged no errors.Earlier rows (2026-11-07, previous head):
GLM-5.3-Flash, bf16 KV on pages, MTP drafts,
--context 131072 --parallel 4, on7c294d557(this code on the previous main) and main267c5af6a; set isTF_GLM_KV_SPLIT=1 TF_KV_POISON=1:"draft": false.kv split: 2 owner classes of 514 pages; a stream takes at most one class (257 pages at the window).free 514 + 0 promised of 514, 514 + 0 of 514.glm-paged-checkoff and set: PASS;glm-index-check: PASS; refusal at 1M x 8 by name.6925e8027): 600 kernels pass gates 1-4; 377 controls red as planted, 0 missed, the eight two-region controls among them. Limit problems are main's at nvcc 13.3 only.What the later parts carry (reference)
The full split already runs on our pair on an older stack (Mia's 1104 recipe,
f350cd3e+ patches, its own two-halves layout): owner-computed decode with no new kernel, prompt twins, carries, #1411's disk states under it; replies byte-identical over 8 request sets, the pool 2.10M → 4.20M tokens, decode −1.6% (1 stream) to −3.6% (8). Numbers and the cache-plan lesson are in the issue thread.By Robert Wojciechowski (GitHub @wojo, X: rwojo).
🤖 Generated with Claude Code