Repository navigation
Conversation
|
Thanks @laudney. We read step 2 early, against the step-1 tree it was built on. Measuring every page first and then rendering each page alone into its page arena reads cleanly. The worker frame now checks each page's index, size and DPI against the selection, and chat's PDF refusals keep their text. We found nothing to ask here. The order stands as you set it out. #2570 is in our landing queue; main has moved under it, so we're rebasing it onto today's main ourselves. Once it's on main, please rebase this onto main, mark it ready, and we'll do the full read with CI.
|
OCR reads PDF pages by index under a page budget sized for one generation per page, and must know every page's size before any inference while holding one rendered page at a time. renderPages takes a page selection and caps from the caller over its running budget, measurePages reports the document's page count and each selected page's size and DPI without a bitmap, the render worker's request carries both and renderer failures take the caller's error kind. Chat PDF input keeps its limits, messages and error kinds.
A Mistral client sends whole PDFs and picks pages with pages, so the route now reads PDF documents with --vision-pdf and the pages list or range string, bounded before it expands and deduplicated under the request's page budget. Every PDF is measured before the first generation, so an index out of range or a request over the budget is refused whole before any inference, and each page then renders alone in its page arena just before it is read, so one rendered page is in memory at a time; a client that leaves renders nothing.
The API guide described an image-only route. It now gives the PDF render limits and DPI, the measuring before inference with one rendered page in memory at a time, the shared page budget, PDF fetches under --vision-pdf, renderer failures, and the pages field; the changelog line names PDFs and the page budget.
842c68e to
92ff8f2
Compare
|
@ashhart rebased onto main
Replaying the old commits conflicted on each one, because they were written against step 1's first version, so I applied the step-2 change to main in three commits instead.
Main has moved since ( |
|
Thanks @laudney, the rebase is just what we asked for. We redid the read here on today's main: the series applies with only the CHANGELOG line's context moved, which we settled by hand, and the OCR, dots.mocr and API boundary steps all pass. Lean still adds nothing. Starting the full CI read now. If it comes back clean this queues with the rest of the step 2 order.
|
document_url/ base64 PDFpages(list or range)Step 2 of the stack #2440 set out. #2570 is merged; this is on main.
run_test.zig"the page budget ... refuse before any generation";mistral_test.zigpagesbounds;pdf_measure_test.zig(a measure over the budget is refused with the render's message)run_pdf_test.zig: through a stand-in render worker, page k renders only after page k-1's sink returned (old code fails it); peak RSS belowexpandBytescalls the unchanged render pathWhat this changes
A Mistral request may now carry a PDF (
--vision-pdf), andpagesselects pages. Before any inference, the request measures each PDF: the worker loads the selected pages and returns each page's size and DPI with no bitmap, so a bad page index, an unloadable page or a request overTENSORFOLD_MAX_PDF_PAGESfails before the GPU runs, and the image budget is known. Each page then renders alone in the page's arena just before its generation and goes with it. The worker request gains ameasureflag; a zero-length page in its reply is accepted only for a measure.484437b6a, with ocr: POST /v1/ocr on dots.mocr for images (#2440 step 1) #2570); 3 commits484437b6a: no conflict (this head is on it)Checks on this head
ocr-step-2), on main484437b6a. Rebuilt on main after ocr: POST /v1/ocr on dots.mocr for images (#2440 step 1) #2570 merged, as you asked: replaying the old commits conflicted on each one (they were written against step 1's first version), so the step-2 change went onto main in three commits with the original messages. Each file's change equals the one on the old head 842c68e except the CHANGELOG line, which now edits main's ocr: POST /v1/ocr on dots.mocr for images (#2440 step 1) #2570 line; the eight review commits' content is inside the three. One addition: the PDF run test's scripted replies use step 2's reader format (text 0 0 10 10), where they used thepagekind that only ocr: GLM-OCR reader on /v1/ocr (#2440 step 3) #2572 adds; with PDFium and libjpeg-turbo present the old test failed here, and without them it skipspython3 tools/zig/lean_check.py, on main484437b6aand on this head: 341 and 341, 0 new.zig build test-ocr-host test-api-boundaries test-server-cpu test-dots-mocr-host check-native-hip-host -Dhip=true: 20/20 steps, 1516 of 1533 pass, 17 skip, 0 fail; the first commit alone passestest-server-cpuandcheck-native-hip-hostzig fmt --check zig/src build.zig zig/buildandgit diff --check: clean. The PDF tests skip without PDFium, sotest-server-cpualso ran with PDFium chromium/7725 and libjpeg-turbo 3.1.4.1: every OCR and PDF test passes;messages"a user message with a PDF file content part ..." fails, and fails the same way on main with these libraries (its rendered-JPEG hashes)Receipt
Main
484437b6aagainst this head, chat's PDF path: dots.mocr on gfx1201, a 2-page PDF as onedata:application/pdfpart, "Read the text in this document.", temperature 0,max_tokens128; prompt ids fromPOST /v1/tokenize, reply ids from"return_token_ids": true, compared element by element:484437b6a92ff8f246main == branch: yes, every id in both lists; the reply token SHA is also the one main
9c9bf542cgave before the rebase.Peak RSS for one
/v1/ocrrequest on a 16-page PDF (dots.mocr), run on the old head before the rebase (the page loop is unchanged): each page a full-page raster that renders to a 1.14 MB JPEG (18.3 MB for 16); fresh server, one warm-up request,clear_refs, thenVmRSSsampled every 20 ms; two runs a build:29ce29393(every page first)What remains on this branch is one page's work (its decoded pixels and the model's image input), which does not grow with the page count.
Client receipt on this head (dots.mocr, gfx1201,
mistralai3.7.0 unmodified): base64 PDF withpages=[0],table_format="html",extract_header=True,include_image_base64=True: 200, 1 page at 144 DPI (1224x1584), header "ACME Corporation - Quarterly Report",tbl-0.html, crop 523x295 equal to its box, 8 typed blocks, 0 unknown. On the old base the crop was 522x295: one box digit moved with main's changes since9c9bf542c, not with this branch (the chat ids above are equal on main and here).Boxes it reaches: the HIP native server with an OCR reader,
--vision --vision-pdf.New switch? No.
TENSORFOLD_MAX_PDF_PAGES(default 16) also bounds an OCR request's pages.Environment: base
484437b6a; Zig 0.17.0; ROCm 10.1 (hipcc HIP 7.16); one Radeon AI PRO R9700 (gfx1201); Linux; dots.mocrdots-studio/dots.mocre539fbb52280393adc081b289ec597430a0f9031, GLM-OCRzai-org/GLM-OCR2e85a62840ccac27daa451df36c736c4636b8628; PDFium chromium/7725; libjpeg-turbo 3.1.4.1. The end-to-end client receipts ran on the whole stack's code (c517f4379, byte-equal to step 6's OCR files), which serves this step's requests through this step's code.Paths and platforms the change reaches, which I ran and which I could not: HIP gfx1201 end to end; chat PDF input shares
pdf.zigandpdf_worker.zig(token ids above); Windows renders in process and was not run.Exactness: n/a; no generation path changes.
Main against this branch, same machine and checkpoint: the chat token ids above.
Decode speed before and after: n/a.
Prompt speed before and after: n/a.
Licence of any new weights or drafter, or "n/a": n/a
Checklist