Repository navigation
Conversation
…x kernels nothing launches q27_prompt_product, _silu, _decay, _qk, q27_prompt_conv and q27_prompt_recur were reached only through prompt_gpu.Kernels and prompt_state_gpu.Kernels, which nothing builds with pipelines. The host records in prompt_glue.zig, prompt_gpu.zig, prompt_state.zig and prompt_state_gpu.zig, the build and runtime source lists, the runtime analysis line and the prompt.metal write in tests/qwen27/run.zig go with them.
…see calls No family calls rotate(); the shared attention's runtime callers (Gemma 4 and Qwen27 DFlash) rope their rows elsewhere. Shape loses rotary_dim and rope, which only the kernel read, and lanes_see feeds raw K and Q straight to append and attention.
…passed to shared_attention; changelog line
…says dsv41_cuda.module pinned the family, kernel and runtime modules to Debug, and dsv41-assets built tf-dsv41-assets on it, so the AOT planner always ran unoptimized. Defilan measured one aot-needs enumeration at --context 524288 on a GB10: about 45 minutes as built, under 7 with the family modules at ReleaseSafe, planning the same 138 variants (#2270). module and runtime keep their Debug signatures for every other caller; moduleAt and runtimeAt take the mode, and dsv41-assets builds the tool on moduleAt(.ReleaseSafe), so its runtime and kernel modules go to ReleaseSafe too. ReleaseSafe keeps the safety checks, and no code under the tool reads the optimize mode, so the plan is unchanged. test-dsv41-assets-host stays on the Debug module. Refs #2270. Co-authored-by: Defilan <7034124+Defilan@users.noreply.github.com>
…nt blocks (relu^2 up, fp32 or bf16 down)
…and bf16 down, gates 1-5 with two controls
…, as pack does, with a test
…Apache-2.0, as its model card now says
Give tf_lane the reg path's 16-row grid axis and runtime sums stride. Each block keeps the same group products, FMAs and K reduction; cooperative and older-chip encodes retain 16 rows. Qwen27 selects a whole-prompt key for non-reg widths behind the existing Ultra gate. Decode and mixed stacks stay short. Compare each width with main's frozen short kernel, including ragged tails, producer sums, strided outputs and three controls. Pin real quant dispatch counts with main-red host regressions. Refs #2818
The full lane contract compiles all four tf_lane_sums sources for M1 and M2 at a 1024-thread pipeline limit. Record those measured values and keep the actual 256-thread launch. Tensor-op sources remain explicitly unmeasured on these chips. Pin the distinction in a host regression; no shader changes. Refs #2818
Only whole-prompt layouts compile the independent row-block header. Short layouts retain a scalar group coordinate and their original row extent; guarded sums remain bounds-safe. Keep older-chip sources on that short variant as well. Compare both short and long programs against frozen main at every maintained byte-check shape and row count. Refs #2818
Leave other affine lane callers on their scalar 16-row source and capacity. Only the Qwen prompt key and byte fixture request independent row blocks. Pin the shared shader bytes, describe the row layout accurately and cover the 16/17-row boundary. Refs #2818
…, read at GLM's open GLM-5.3-Flash's attention keeps no rotary positions, so only its trained length bounds the window. TF_GLM_CONTEXT_CEILING, a .glm row, sets the window --context may ask for, up to 2^31 - 32, where the slots keep i32 positions. GLM reads it at its open once its .glm rows are checked; unset, the ceiling and the refusal are main's. A window past the trained length says so at startup.
…a_env.zig cuda_native.zig had passed the 600-line limit. ceiling_switch and raisedCeiling move to cuda_env.zig, which owns the .glm check, and the i32 test to cuda_env_test.zig; cuda_native.zig keeps the two names the CUDA open looks up.
…est file and cuda_window.zig Main grew cuda.zig to 587 lines, so the branch's additions passed the 600-line limit again. The open-time test of raised context ceilings moves to cuda_ceiling_test.zig, imported beside cuda_health_test.zig, and the lookup of a family's raised ceiling and its switch to cuda_window.zig's familyCeiling; cuda.zig keeps one call.
…t.zig Main's eb29a68 added a line to cuda_native.zig, so merged with main the file comes to 601 lines, past the 600-line limit. The test that capacity keeps a verify window past the context, a multiple of 64, moves whole to cuda_pages_test.zig, which tests GLM's pages at that capacity: 594 lines in cuda_native.zig here, 595 merged with main.
A fraction after --compact-at auto left auto mode on, because the flag set compact_auto and a later fraction never cleared it, so the server kept the first value where every other flag keeps the last. Each occurrence now replaces both fields, which a config file's value under a command-line override depends on, and a filtered host step runs the test that fails without the change.
Pin eviction of redundant own history before another conversation's newest state under pressure. Retain prompt-born provenance and prove a same-prompt regenerate restores history when all three states fit. Refs #2881
Give test-native a timed shard with the Metal toolchain. Keep it nightly until the CI runner establishes its wall time. Refs #2880
…xperts, Drama 3 and Kolibri-1 read through it
…t says an f16 NaN stays a NaN
…er, its keys unchanged
The video byte gate, tf-glm-video with glm_video_fixtures.py, builds only on macOS and needs a checkpoint's processor. This tool needs only a libtfvideo build and clips: it decodes every frame, GLM's 2 frames a second, and the first three and last frames, hashes each frame's RGB rows, and exits nonzero when any frame or return code differs from PyAV's or from a second build's. A clip that will not open or decode fails and the run goes on. --time-only times the decoder alone. Nothing in the engine or the build calls it, so no served path changes.
…links FFmpeg's x86_64 assembly addresses tables defined in C, such as ff_pw_* and ff_h264_cabac_tables, with 32-bit PC-relative relocations. Linked from the static libav*.a into a shared library, those tables are exported and so preemptible, and the link stopped on R_X86_64_PC32 errors (5,046 with gcc 13.3) although build_ffmpeg.sh passes --enable-pic. On ELF targets the library now links with a version script that exports the four tf_video_* symbols core/tfvideo.zig looks up and makes every other symbol local, so the tables bind inside the library with no text relocations. check-video-exports-linux checks this for aarch64 and x86_64 Linux with a planted table. The loader, build_ffmpeg.sh and its FFmpeg pin, and Mach-O builds are unchanged.
engine_api.zig is at the 600-line limit, and the OCR reader types that follow add to it. Memory and MemorySource describe a backend's status, as HealthSource and Status do, so they move beside them in engine_status.zig and engine_api re-exports them under the same names; no caller changes.
The libjpeg-turbo loader and the BGRA page encoder move to their own module, jpeg_out.zig, with the optional-library loader that PDFium shares, so the OCR route can encode image crops and pdf.zig gains the room its page selection needs later; jpegPixels becomes jpegSize, which returns the width and height a frame header names, for the OCR page planner. Chat PDF input behaves exactly as before, and the new encodeRgb is covered by a round-trip test.
An engine now declares an optional OCR reader in Info.ocr: the prompt that follows the page image, the geometry it gives and a parse from generated text to Markdown blocks with quads in original pixels. PreparedImage gains the tower-input frame so readers can map boxes back, the HIP offer carries the reader through, and a bounded reader for the HTML table subset dots.mocr writes turns tables into GFM pipes, clean spanned HTML or text rows, escaping literal markup so Markdown never reinterprets recognized text.
The OCR route reuses chat.prepare and chat.generate but needs its own encoded-byte limits for page images, a reply that may fill the room the prompt leaves, and the frame each image was resized to so readers can map boxes back. chat.Input gains image_bytes and fill_room with defaults equal to chat's values, a route with its own limits decodes data URLs against them and its refusals name them while chat keeps its words, and the frame of each image part flows from PreparedImage through Expanded and Rendered to Prepared.
Every OCR reply needs the same steps between a reader's blocks and its body, so they live once under server/ocr: the request fault and source types, quad validation with Mistral's integer box and PaddleX's polygon, Markdown to plain lines with a table finder, page sizes from image headers, and the page and image-encode budgets. Every scan is linear in its input, Conn.sendJsonWith adds headers such as Retry-After, and the new test-ocr-host step runs the OCR tests without a GPU.
The OCR route needs each OCR family to say how it reads a page, so dots.mocr now offers an Ocr reader: the model card's layout prompt, block geometry, and a parser from the model's JSON block list to normalized Markdown blocks that maps each category to a kind, scales each box from the tower frame to the original image and turns HTML tables into pipe tables. Its prepare path reports the frame, greedy Hugging Face oracle outputs pin the prompt and the parser, and golden boxes on a 1000 x 333 page check the stretch back to client pixels under two token budgets, and a page past 16 megapixels is refused from its header.
Mistral's OCR SDKs need a server that reads a document and answers in their response shape, so POST /v1/ocr now decodes the Mistral request, runs one greedy generation for its page image through the family's OCR reader with the same prompt ids a chat request gets, and writes the page with Markdown, tables, image crops and blocks under one request-wide page and encode budget. Annotations, file documents, PDF documents and pages are refused by name; the Python SDK's captured request is replayed and its reply checked against the SDK's models, and the route needs the inference scope.
Clients need the route's contract before they can point the Mistral SDKs at the server, so docs/api.md gains an OCR section: the route and the reader, the Markdown the reader writes, page and image budgets, the refusals of PDF documents and pages, failure statuses and every Mistral field, and the inference scope lists /v1/ocr. The changelog names the route under Unreleased.
Step 1 reads page images through /v1/ocr only, so the code that later steps use moves to them: the plain-text line path in markdown.zig, the keep_math switch it needed, rowsText, PaddleX's polygon with its rounding helpers, OcrLine and fetch.https's PDF switch all return in the step that first calls them. The pipe-table tests that went through plainLines now read the cell back through pipeTable, which keeps the emphasis, code, link, entity and one-pass guards on the clean-up that step 1 still runs, and two comments now cite docs/api.md sections that exist.
The run decoded a page up to 32 Mi pixels, GLM-OCR's limit, but step 1 serves only dots.mocr, whose processor stops at 16 Mi pixels. The page limit is now picture.max_pixels, so a larger page is refused as bad input at decode with the picture's own message, before the reader runs; GLM-OCR's step raises it again.
Step 1 reads one page image a request, so the page budget, the planned page's source, index and DPI, the sink's source argument, Geometry with its none and line readers, Kind.page, the null quad, the no-geometry header and footer refusal and the Scanner's line payloads have no caller here; each returns with the step that first uses it. An image is page 0 at 72 DPI, every block has a box, and comments that named PDF, /ocr, PaddleX or visualisations now describe this step only.
A route test sends a PNG whose header names 4097 x 4096 pixels, 4096 past the 16 Mi decoded-pixel limit and under 8192 a side, and checks HTTP 400 with the decoder's own message before any generation. The server started without --vision now checks the refusal's text, not only its status, and that nothing was generated.
The reader prompt is the model card's text and the demo fixtures are the model's output on the publisher's image, so the fixture README and a THIRD_PARTY_NOTICES.md entry now name the publisher's MIT License, Copyright (c) 2025 rednote-hilab, and the dots.mocr License Agreement that adds terms to it. LICENSES/ holds both texts: the MIT file unchanged from the pinned commit and the agreement from the pinned release with three lines' trailing spaces removed.
Step 1 set a source's name but never read it; docTR in step 5 is the first reader that names its sources in replies. The field and its writers in the Mistral request and the run tests go, and step 5 brings the field back with its first reader, as the review asked.
The dots.mocr License Agreement covers the model card's prompt and the model's outputs, and its terms should not enter the tree. The reader now uses a prompt written for TensorFold (prompt.txt), the fixtures record oracle runs on two pages the tool draws, and they store token-id and text hashes, not the model's text. The tests read written block lists, and TF_DOTS_OCR_DIR checks real oracle runs by hash before it parses them. The licence copies and notice go.
Resolving docs/api.md during the rebase onto 33ada77 kept only the inference scope row and lost the OCR section (route and reader, Reader Markdown, pages and budgets, failures, Mistral fields) that code comments, the CHANGELOG line and the PR text cite; this puts it back, unchanged, between Rerank and System One.
The rebase onto 33ada77 joined #2253's image-only refusal with /v1/ocr in routes.zig by counting OCR as a text route; this test sends POST /v1/ocr to the images suite's image-only stub and expects the same 400 naming the images route, with no engine call. It fails when is_ocr is taken out of is_text.
Owner
Author
|
Conflicts with main (main moved after stacking); restacking. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2868 #2875 #2723 #2902 #2903 #2906 #2861 #2570 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.