Repository navigation
Conversation
…e the rank stops Split's start-up warm-up sends and receives 4 MiB with every other rank, and runs before Engine.init's split probe, whose refusal names the fix. On a fabric where some ranks reach only their neighbours (a ring of four without a switch) the warm-up's group ends with an NCCL error first, and check stops every rank at once, so the probe's refusal never prints. From four ranks, where a ring has ranks that are not neighbours, the warm-up now prints that refusal, naming gather, when its group end fails under split, and hands check a name that starts with it, so a rank that stops on another rank's failure names the refusal as the cause too. Fewer ranks, ring and gather warm-ups, and a warm-up that finishes, print nothing new.
P2 of #2444 adds reuse identity (C6) beside the tree's words, so the store's begin and keep now take a Query instead of a bare prompt. The engine_api side carries every identity field with defaults and reads only the key for now; the core side carries the key and prompt length, which the CUDA adapter converts as it already does for Plan. Every caller passes the same words as before, so no store user changes behaviour.
The checkpoint manifest of #2444 (C3) lets several nodes and entries name one page, so the tree now holds its pages through Holds, a registry that takes one pool reference on a page's first name and gives it back on the last. The new prompt_radix_parts.zig also declares the manifest's types for the next steps. Node pages are distinct today, so the pool sees the same retains and releases in the same order and every store user is unchanged.
A family whose cache has several prefix groups (a target and a drafter's full attention on one pool, #2444 C3) needs one page id of each group for every logical page on the tree's path. The tree now keeps per ids a logical page, page-major, counts logical pages in its walks and ids in its limits, and initGroups declares the groups, refusing groups with other rows a page than the store's. With one group, today's case, every index and count is as before.
C3 of #2444: a family may declare checkpoint groups whose pages an entry holds beside its legacy snapshot. A copied part takes store-held pool pages its live bytes are copied into; a retained part names the stream's own pages and copies a partial last page. Entries are published only whole, with every copy fenced before a page goes back; a node is pinned while a keep builds it or a resume restores it, so reclaim never frees its own source; reclaim first frees idle parts. Also fixes a leak in insert when its second allocation failed. Stores without parts are unchanged.
C3 of #2444 defines the store for a family with no checkpoint at all (pure full attention, as vLLM's prefix cache): it keeps the path's pages with no snapshot slot, resumes at the deepest page edge a prompt repeats, inside a node too, and makes room by freeing idle leaves, never the path a pass resumes. No family on the store is such a family today, so every store user is unchanged; the first one (MiniMax M3, P7) needs no store change.
A mark whose path and checkpoint parts passed the page limit was planned anyway; making room for it evicted a kept checkpoint before the keep itself was refused. Planning now counts the part pages it knows before the keep (copied parts and partial retained pages), so such a mark is refused up front. Long trailing comments in the lines this branch added move above their lines.
The CUDA ring families' snapshot saves take pool pages, so a save can make the pool reclaim from the store in the middle of a keep. On a keep onto a leaf an earlier failed save left with no entry, that reclaim could prune the very leaf the keep then writes its entry to. The keep's node pin prevents it; this test holds a save that needs one page more than free over exactly that leaf, and it crashes with the pin removed.
…ckground pass's chunks narrow with depth A 20 GiB prompt cache holds two 128k states. The state kept at a reply's end pushed out the one at the history end, so a client whose replayed reply differs from the generated one resumed a turn further back. Slot 0's in-slot rewind buffer now exists whenever there is room, also beside a cache that keeps two full-context states, where needsRewind alone makes none, and its point is carried through passes that only extend slot 0, so the next turn rewinds in place. Beside such a wide cache, which keeps the reply's end, the warm pass over slot 0 runs when it carries that point; beside a smaller cache it still yields, as on main. Background chunks scale down with depth so a waiting request is held for less.
…'s choice are tested rules Standing.warmYields holds the warm pass's yield in opening, Standing.take holds takeStanding's use of the record and its carried rewind point, and makesRewind holds rewindInit's choice: the buffer wherever room holds it, tight where needsRewind says the cache cannot keep two full-context states, else wide. Each has a test that fails when its rule is reverted. The changelog takes the line for #2723 under a new Unreleased heading.
… new rewind point Slot 0 remembers the rows of the reply it generated. A follow-up whose prompt replays that reply as generated and resumes, by the slot or a cache state, inside it or at its end skips the extra rewind stop at its history end, so its fill is the same as main's. A client that drops the reply's reasoning, a cold turn and a resume that goes back still make a fresh rewind point.
check_cuda_sources.py failed 66 of its 918 checks on main: eleven units use SDK names that the fixed declarations in tools/zig/cuda_host lack. Declare each as CUDA 13.0 does: __syncthreads_or and __double2float_rz from sm_20_intrinsics.h, __float2int_rn from crt/device_functions.h, __float2bfloat16_rz from cuda_bf16.h, __hmul on half from cuda_fp16.h, ldexpf from crt/math_functions.h, and the __maxnreg__ macro from crt/host_defines.h, whose maxnreg attribute Clang ignores with a warning. Only the check reads these headers: no kernel, engine or test binary changes.
…er, its keys unchanged
The video byte gate, tf-glm-video with glm_video_fixtures.py, builds only on macOS and needs a checkpoint's processor. This tool needs only a libtfvideo build and clips: it decodes every frame, GLM's 2 frames a second, and the first three and last frames, hashes each frame's RGB rows, and exits nonzero when any frame or return code differs from PyAV's or from a second build's. A clip that will not open or decode fails and the run goes on. --time-only times the decoder alone. Nothing in the engine or the build calls it, so no served path changes.
…links FFmpeg's x86_64 assembly addresses tables defined in C, such as ff_pw_* and ff_h264_cabac_tables, with 32-bit PC-relative relocations. Linked from the static libav*.a into a shared library, those tables are exported and so preemptible, and the link stopped on R_X86_64_PC32 errors (5,046 with gcc 13.3) although build_ffmpeg.sh passes --enable-pic. On ELF targets the library now links with a version script that exports the four tf_video_* symbols core/tfvideo.zig looks up and makes every other symbol local, so the tables bind inside the library with no text relocations. check-video-exports-linux checks this for aarch64 and x86_64 Linux with a planted table. The loader, build_ffmpeg.sh and its FFmpeg pin, and Mach-O builds are unchanged.
engine_api.zig is at the 600-line limit, and the OCR reader types that follow add to it. Memory and MemorySource describe a backend's status, as HealthSource and Status do, so they move beside them in engine_status.zig and engine_api re-exports them under the same names; no caller changes.
The libjpeg-turbo loader and the BGRA page encoder move to their own module, jpeg_out.zig, with the optional-library loader that PDFium shares, so the OCR route can encode image crops and pdf.zig gains the room its page selection needs later; jpegPixels becomes jpegSize, which returns the width and height a frame header names, for the OCR page planner. Chat PDF input behaves exactly as before, and the new encodeRgb is covered by a round-trip test.
An engine now declares an optional OCR reader in Info.ocr: the prompt that follows the page image, the geometry it gives and a parse from generated text to Markdown blocks with quads in original pixels. PreparedImage gains the tower-input frame so readers can map boxes back, the HIP offer carries the reader through, and a bounded reader for the HTML table subset dots.mocr writes turns tables into GFM pipes, clean spanned HTML or text rows, escaping literal markup so Markdown never reinterprets recognized text.
The OCR route reuses chat.prepare and chat.generate but needs its own encoded-byte limits for page images, a reply that may fill the room the prompt leaves, and the frame each image was resized to so readers can map boxes back. chat.Input gains image_bytes and fill_room with defaults equal to chat's values, a route with its own limits decodes data URLs against them and its refusals name them while chat keeps its words, and the frame of each image part flows from PreparedImage through Expanded and Rendered to Prepared.
Every OCR reply needs the same steps between a reader's blocks and its body, so they live once under server/ocr: the request fault and source types, quad validation with Mistral's integer box and PaddleX's polygon, Markdown to plain lines with a table finder, page sizes from image headers, and the page and image-encode budgets. Every scan is linear in its input, Conn.sendJsonWith adds headers such as Retry-After, and the new test-ocr-host step runs the OCR tests without a GPU.
The OCR route needs each OCR family to say how it reads a page, so dots.mocr now offers an Ocr reader: the model card's layout prompt, block geometry, and a parser from the model's JSON block list to normalized Markdown blocks that maps each category to a kind, scales each box from the tower frame to the original image and turns HTML tables into pipe tables. Its prepare path reports the frame, greedy Hugging Face oracle outputs pin the prompt and the parser, and golden boxes on a 1000 x 333 page check the stretch back to client pixels under two token budgets, and a page past 16 megapixels is refused from its header.
Mistral's OCR SDKs need a server that reads a document and answers in their response shape, so POST /v1/ocr now decodes the Mistral request, runs one greedy generation for its page image through the family's OCR reader with the same prompt ids a chat request gets, and writes the page with Markdown, tables, image crops and blocks under one request-wide page and encode budget. Annotations, file documents, PDF documents and pages are refused by name; the Python SDK's captured request is replayed and its reply checked against the SDK's models, and the route needs the inference scope.
Clients need the route's contract before they can point the Mistral SDKs at the server, so docs/api.md gains an OCR section: the route and the reader, the Markdown the reader writes, page and image budgets, the refusals of PDF documents and pages, failure statuses and every Mistral field, and the inference scope lists /v1/ocr. The changelog names the route under Unreleased.
Step 1 reads page images through /v1/ocr only, so the code that later steps use moves to them: the plain-text line path in markdown.zig, the keep_math switch it needed, rowsText, PaddleX's polygon with its rounding helpers, OcrLine and fetch.https's PDF switch all return in the step that first calls them. The pipe-table tests that went through plainLines now read the cell back through pipeTable, which keeps the emphasis, code, link, entity and one-pass guards on the clean-up that step 1 still runs, and two comments now cite docs/api.md sections that exist.
The run decoded a page up to 32 Mi pixels, GLM-OCR's limit, but step 1 serves only dots.mocr, whose processor stops at 16 Mi pixels. The page limit is now picture.max_pixels, so a larger page is refused as bad input at decode with the picture's own message, before the reader runs; GLM-OCR's step raises it again.
Step 1 reads one page image a request, so the page budget, the planned page's source, index and DPI, the sink's source argument, Geometry with its none and line readers, Kind.page, the null quad, the no-geometry header and footer refusal and the Scanner's line payloads have no caller here; each returns with the step that first uses it. An image is page 0 at 72 DPI, every block has a box, and comments that named PDF, /ocr, PaddleX or visualisations now describe this step only.
A route test sends a PNG whose header names 4097 x 4096 pixels, 4096 past the 16 Mi decoded-pixel limit and under 8192 a side, and checks HTTP 400 with the decoder's own message before any generation. The server started without --vision now checks the refusal's text, not only its status, and that nothing was generated.
The reader prompt is the model card's text and the demo fixtures are the model's output on the publisher's image, so the fixture README and a THIRD_PARTY_NOTICES.md entry now name the publisher's MIT License, Copyright (c) 2025 rednote-hilab, and the dots.mocr License Agreement that adds terms to it. LICENSES/ holds both texts: the MIT file unchanged from the pinned commit and the agreement from the pinned release with three lines' trailing spaces removed.
Step 1 set a source's name but never read it; docTR in step 5 is the first reader that names its sources in replies. The field and its writers in the Mistral request and the run tests go, and step 5 brings the field back with its first reader, as the review asked.
The dots.mocr License Agreement covers the model card's prompt and the model's outputs, and its terms should not enter the tree. The reader now uses a prompt written for TensorFold (prompt.txt), the fixtures record oracle runs on two pages the tool draws, and they store token-id and text hashes, not the model's text. The tests read written block lists, and TF_DOTS_OCR_DIR checks real oracle runs by hash before it parses them. The licence copies and notice go.
Resolving docs/api.md during the rebase onto 33ada77 kept only the inference scope row and lost the OCR section (route and reader, Reader Markdown, pages and budgets, failures, Mistral fields) that code comments, the CHANGELOG line and the PR text cite; this puts it back, unchanged, between Rerank and System One.
The rebase onto 33ada77 joined #2253's image-only refusal with /v1/ocr in routes.zig by counting OCR as a text route; this test sends POST /v1/ocr to the images suite's image-only stub and expects the same 400 naming the images route, with no engine call. It fails when is_ocr is taken out of is_text.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2868 #2875 #2723 #2902 #2903 #2906 #2861 #2570 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.