Repository navigation
Conversation
… run on pushes to main only A landing train pushes one commit a PR, and every push to main kept its own full No GPU run: seven runs in four minutes on 13 Nov, each a tree the train's own run had already tested, queued ahead of the next train's run. Main pushes now share one group that a newer push cancels, as a pull request's do; the nightly schedule keeps its own group. The app workflow's push trigger, which had no branch filter, now runs on main only, as its pull_request runs cover branches.
… bf16 logits on request tf_route_logits gave each lane four 32-wide slices of k, which covers a hidden width of exactly 4096 (GLM-5.3-Flash and Step-5); at 2048 it read past each row and past the packed router. Each lane now takes hidden / 1024 slices, the same four at 4096, so those routers' arithmetic is unchanged, and `source` refuses a width that is not a multiple of 1024. `round_logits` rounds each logit to bf16 before the select, as a bf16 router matmul (MLX's) gives it.
A Metal family for `model_type: flint` (RecursiveSI/flint), served from Recursive's MLX INT4 export (RecursiveSI/flint-int4: affine 4-bit groups of 64, an 8-bit embedding) by the lane host with the prompt cache, structured output, tools and several streams. - Layers: 35 sliding (513-token window, RoPE) and 14 full (no positional encoding) GQA layers, 32 query heads over 4 key heads, per-head q/k norms and a learned sink each head; two dense SwiGLU layers, then 256 routed experts (top 6 by sigmoid score, renormalized, no score bias) beside an ungated shared expert. - Rounding: MLX's points, as RecursiveSI's rec_mlx computes the model: a bf16 residual, rms_norm as w * bf16(x * inv), SiLU rounded op by op, bf16 router logits. - Attention: prompt chunks on the tensor units (gemma4's prompt kernel over a position-major cache, the sink in the denominator); decode rows on sdpa_vector's lanes over key blocks at absolute positions, merged with the sink by MLX's second pass, so a row's bits do not depend on the rows beside it. Sliding layers keep a ring of the window and one prompt chunk. - Projections: q|k|v and each MLP's gate|up stacked into one matrix at load (one launch, the same row sums); one launch norms and rotates q and k and writes k and v; residual adds fused with the next norm. - Routing: core moe_route's logits, then a top-k pick a simdgroup a row (the core select's arithmetic without its per-Mac lists). - Serving: several streams' windows share one forward, each reply equal to its run alone; a lone stream's steps run a step ahead, drawn on the GPU (argmax, or the keyed sampler of core lanes' gpu_rule and gpu_full at Flint's vocabulary) into a token ring, and the host draws with the same sampler's twin. - tf-flint-run: greedy tokens, every position's logits, forced steps and step timing, for a reference's check, and one-row steps against windows of any width, bit for bit (`exact`).
…ac's RAM is refused, not converted
…(the fp32 form missed MLX on 1,360 of 65,280 bf16 gates, and gave 0 below -17); the router keeps MLX's fp32 form
…al_main has room under 600 lines
…p, decode and prompt attention; gates 1-4 tests, 16 planted controls, gate-5 records), and core tf_route_logits at Flint's, GLM's and Step-5's widths
…he tensor units keep (fp32 cut to 10 mantissa bits, measured on the device before the gates run), not bf16's
A recorded one-row step copied each stream's whole GDN state and conv tail for a keep of no rows, and served traffic never keeps one. Part now carries undo (default on, so warm-up, qualification and host tests keep their copy); the served queue and verify parts set it only while learning. With undo off the staged segment reads no before, recurrent.prepare leases no copy, and the two kernels skip only their copy: the conv tail's in-place update still runs. An empty keep after such a step refuses with QwenNoRollbackCopy rather than restoring bytes the step never wrote. Host: qwen35 shared, cache, pending, rounds, verify, live, prefill and the recurrent workspace steps pass; compile-cuda-tests for Linux compiles.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #3230 #1177 #2933 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.