Repository navigation
Conversation
… run on pushes to main only A landing train pushes one commit a PR, and every push to main kept its own full No GPU run: seven runs in four minutes on 13 Nov, each a tree the train's own run had already tested, queued ahead of the next train's run. Main pushes now share one group that a newer push cancels, as a pull request's do; the nightly schedule keeps its own group. The app workflow's push trigger, which had no branch filter, now runs on main only, as its pull_request runs cover branches.
… bf16 logits on request tf_route_logits gave each lane four 32-wide slices of k, which covers a hidden width of exactly 4096 (GLM-5.3-Flash and Step-5); at 2048 it read past each row and past the packed router. Each lane now takes hidden / 1024 slices, the same four at 4096, so those routers' arithmetic is unchanged, and `source` refuses a width that is not a multiple of 1024. `round_logits` rounds each logit to bf16 before the select, as a bf16 router matmul (MLX's) gives it.
A Metal family for `model_type: flint` (RecursiveSI/flint), served from Recursive's MLX INT4 export (RecursiveSI/flint-int4: affine 4-bit groups of 64, an 8-bit embedding) by the lane host with the prompt cache, structured output, tools and several streams. - Layers: 35 sliding (513-token window, RoPE) and 14 full (no positional encoding) GQA layers, 32 query heads over 4 key heads, per-head q/k norms and a learned sink each head; two dense SwiGLU layers, then 256 routed experts (top 6 by sigmoid score, renormalized, no score bias) beside an ungated shared expert. - Rounding: MLX's points, as RecursiveSI's rec_mlx computes the model: a bf16 residual, rms_norm as w * bf16(x * inv), SiLU rounded op by op, bf16 router logits. - Attention: prompt chunks on the tensor units (gemma4's prompt kernel over a position-major cache, the sink in the denominator); decode rows on sdpa_vector's lanes over key blocks at absolute positions, merged with the sink by MLX's second pass, so a row's bits do not depend on the rows beside it. Sliding layers keep a ring of the window and one prompt chunk. - Projections: q|k|v and each MLP's gate|up stacked into one matrix at load (one launch, the same row sums); one launch norms and rotates q and k and writes k and v; residual adds fused with the next norm. - Routing: core moe_route's logits, then a top-k pick a simdgroup a row (the core select's arithmetic without its per-Mac lists). - Serving: several streams' windows share one forward, each reply equal to its run alone; a lone stream's steps run a step ahead, drawn on the GPU (argmax, or the keyed sampler of core lanes' gpu_rule and gpu_full at Flint's vocabulary) into a token ring, and the host draws with the same sampler's twin. - tf-flint-run: greedy tokens, every position's logits, forced steps and step timing, for a reference's check, and one-row steps against windows of any width, bit for bit (`exact`).
…ac's RAM is refused, not converted
…(the fp32 form missed MLX on 1,360 of 65,280 bf16 gates, and gave 0 below -17); the router keeps MLX's fp32 form
…al_main has room under 600 lines
…p, decode and prompt attention; gates 1-4 tests, 16 planted controls, gate-5 records), and core tf_route_logits at Flint's, GLM's and Step-5's widths
…he tensor units keep (fp32 cut to 10 mantissa bits, measured on the device before the gates run), not bf16's
A recorded one-row step copied each stream's whole GDN state and conv tail for a keep of no rows, and served traffic never keeps one. Part now carries undo (default on, so warm-up, qualification and host tests keep their copy); the served queue and verify parts set it only while learning. With undo off the staged segment reads no before, recurrent.prepare leases no copy, and the two kernels skip only their copy: the conv tail's in-place update still runs. An empty keep after such a step refuses with QwenNoRollbackCopy rather than restoring bytes the step never wrote. Host: qwen35 shared, cache, pending, rounds, verify, live, prefill and the recurrent workspace steps pass; compile-cuda-tests for Linux compiles.
POST /v1/requests/{id}/cancel finds a request by the chatcmpl-, cmpl- or resp_ id its client got and sends the
engine the cancel a disconnect sends; the reply ends "cancelled" with the tokens it had, a queued one with none, and
other streams are unchanged. 404 request_not_found for an id no request holds.
The ids are held for the whole client request in openai.runNamed, and the ask rides on chat.Input, so every
generation for that request reads it (compaction's reply, Cortheon's rounds) and server-made ones never do. A
Cortheon turn stops after a cancelled round. A Responses reply ends with status "cancelled" in its
response.incomplete event, without completed_at, and a Responses socket keeps it like an incomplete one.
Who may cancel: with API keys, a full or [admin] key (the key gate refuses scoped keys, and the route checks the
matched key again). Without keys, only a caller whose socket is on loopback: the peer address from accept(), never
a header, including listener.peerText's IPv4-mapped form, which IpAddress.parse refuses. A keyless request that
carries Forwarded, X-Forwarded-For, X-Forwarded-Host, X-Real-IP, Via, CF-Connecting-IP or True-Client-IP is refused,
since a reverse proxy on the same machine makes every caller loopback. Refusals answer 403 loopback_only.
Fixes #3172
A decode window of 4 rows was captured at 16 rows, so lane_gemv loaded and stored 12 padding rows and attention launched their dead blocks. Shape.init now buckets 1 row as 1, 2-4 as 4, 5-8 as 8, and larger windows as before. Every kernel takes its live rows from the staged segment table and MMA rows are independent, so a row keeps the bits of the same one-row step. Warm-up and the cost table follow the new keys: seven buckets instead of five warm at start, and the live-width tests pin the new call counts. The shapes tests now run in the qwen35 module and get their own host step.
…links FFmpeg's x86_64 assembly addresses tables defined in C, such as ff_pw_* and ff_h264_cabac_tables, with 32-bit PC-relative relocations. Linked from the static libav*.a into a shared library, those tables are exported and so preemptible, and the link stopped on R_X86_64_PC32 errors (5,046 with gcc 13.3) although build_ffmpeg.sh passes --enable-pic. On ELF targets the library now links with a version script that exports the four tf_video_* symbols core/tfvideo.zig looks up and makes every other symbol local, so the tables bind inside the library with no text relocations. check-video-exports-linux checks this for aarch64 and x86_64 Linux with a planted table. The loader, build_ffmpeg.sh and its FFmpeg pin, and Mach-O builds are unchanged.
…e threads tf_video_frames asked FFmpeg for slice threads only, so H.264 in one slice a picture and MPEG-4 decoded on about one core. MPEG-4, and H.264 that declares no B-frames, now use up to TF_FRAME_THREADS (4) frame threads, none on one CPU. An H.264 packet goes to them only as one I or P slice, with no new extradata to change how its NAL lengths read; anything else returns the decode to main's path, as FFmpeg's frame threads conceal damaged B-frame H.264 unlike main's, with no error or mark to restart on. On a decode error, a frame marked damaged, or a packet with no frame, the decode restarts on main's path from the first undelivered index. A frame waits until every earlier packet has given an unmarked frame.
…MPEG-4, under Unreleased
…iles the page has dashboard_test.zig was wired only into the macOS test step, so no CI job ran it. Since #2655's landing it has expected a "Paged prefix cache" label that dashboard.html never had and nothing in /stats reports, so it failed on every Mac running zig build test. The test now runs from build.zig's shared test step on every OS, and that one string is gone; every other label it checks is on the page.
…softmax body from one source (#2251)
… from one source (#2251)
T0's group backend could not abort GLM's communicator before the RoCE group was installed, because it only borrowed it. GLM now hands the backend the group that owns the communicator from Engine.init, so an abort always reaches it once, and the start-up unwind uses the group's guarded destroy, which skips the destroy after an abort.
Owner
Author
|
Restacking: the train this one was stacked on did not land as tested. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2949 #2918 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.