[BobClawblaw]

Fixes #2073.

This serves Baidu's BF16 baidu/Unlimited-OCR-2 (revision 0480966938291445591f69f08a13b041c073a96a) natively on Metal, as published: the engine reads the release's safetensors directly, with no converted pack and no Python or libtorch in the served engine. The image tower (SAM ViT-B, its neck and CLIP-L), the decoder (page2d rotary, DASH block-sparse attention, a 64-expert MoE) and the publisher's page-guarded greedy decoding run in Zig and Metal. Its caches stay in the model's precision: bf16 keys and values, fp32 router keys, as in the publisher's code.

The SAM encoder, the decoder's attention, its page2d rotary and its MoE gate are written from the papers, the checkpoint's config and the release's modules measured through their inputs and outputs (forward hooks on its own infer, in tools/zig/uocr_reference.py). No code follows _sparse_base_impl.py or the libraries deepencoder.py cites, and none of the earlier code that did is in this branch's history. THIRD_PARTY_NOTICES names only the release's own MIT files the family follows.

	Before (main)	After (this PR)
baidu/Unlimited-OCR-2 (BF16) on Metal	not supported	served by tensorfold-native serve --vision on M5 (tensor units)
Transcribed text vs the release's own infer (torch bf16, CPU) on the five single pages	—	identical on three; on paper-3 and paper-10 it departs only at ties (the same two contenders on both sides, both margins within one bf16 step), which tf-uocr-decode now gates
The five pages as one document	—	reported beside the claim, outside the one-step policy: two of its four departures (steps 300 and 431) are the same two contenders but two and three bf16 steps apart on our side, with float32 on our side of both (see below)
Each image-encoder step alone vs float32 (tf-uocr-blocks)	—	every SAM block, the neck, CLIP's front, every CLIP layer and the projector within twice torch bf16's distance, at most 0.53 of the limits
Repeats and two requests at once vs a lone request	—	identical replies
serve --vision MODEL	—	the family's chat template built in (the release ships none); a checkpoint's own template or --chat-template takes its place. One helper, server/hf_text_parse.zig's builtIn, now holds both built-in templates: Nemotron Parse's replaces the checkpoint's, this family's fills an empty one
--parallel above 1	—	refused: "Unlimited-OCR-2 serves one stream: serve it without --parallel"
Prompt cut into chunks vs whole	—	bit-identical logits and replies (tf-uocr-decode --chunk, odd sizes)
Kernel contract (Metal)	—	24 kernels (12 image encoder, 12 decoder), 24 launch records, through 30 gate cases (a case per dispatch form: uv_attention for SAM and CLIP, uv_im2col for three convolutions, ud_attend for prompt rows and steps, ud_norm alone and with its two additions): SUMMARY 30 kernels: 30 pass gates 1-4, 0 fail, 0 known findings; 30 controls: 30 red as planted, 0 missed; 0 limit problems (tf-kernel-contract --only uocr)
Lean checker	—	386 problems, as main has; 0 new. tests/contract/metal_main.zig 521 lines, server/hf_text.zig 600
--context past the window	—	refused by name before the checkpoint is read (--context 131073: Unlimited-OCR-2 runs to 131072 tokens), tested in native/uocr.zig
Existing models	—	unchanged: no shared kernel or path changes (see below)

Results (M5 Ultra; another model's server loaded on the same GPU, idle; recipe harness ocr plan, head c7cd2ff)

Pages	Prompt tokens	Reply tokens	First token	Decode	Text vs reference	Box coordinates moved
report-1	909	198	0.48 s	299.4 tok/s	identical	2 / 16 (at most 2)
report-2	909	141	0.28 s	301.1 tok/s	identical	0 / 16
paper-1	909	543	0.28 s	297.1 tok/s	identical	1 / 44 (1)
paper-3	909	911	0.28 s	301.1 tok/s	citations written as $ [5, 6, ...] $	1 / 44 (1)
paper-10	909	2,004	0.29 s	298.3 tok/s	table headers written in LaTeX	0 / 28
all five as one document	4,525	3,767	1.70 s	282.5 tok/s	differs; 34 boxes against 35	98 / 136 (at most 824)

The claim is the five single pages. tf-uocr-decode feeds torch bf16's own reply back a token a step and, where our pick differs, now requires a tie: the same two contenders on both sides (the reference's top2, each step's two highest logits, recorded by uocr_reference.py dump) with both margins within one bf16 step at the logits' size; any other departure fails the tool. On this head every single-page departure is a tie (torch's margin, then ours, in bf16 steps):

Page	Step	Contenders	Torch bf16	Ours
report-1	10	2892, 4774	equal (argmax takes 2892)	4774 by 1
report-1	33	27738, 29037	equal	29037 by 1
paper-1	88	10410, 10669	equal	10669 by 1
paper-3	145	764, 223	764 by 1	223 by 0
paper-3	557	26670, 27343	equal	27343 by 1
paper-10	67	7233, 3289	7233 by 1	3289 by 0

report-1's and paper-1's two tokens render the same text, so their transcripts are identical; report-2 has no departure. The five-page document is reported beside the claim, outside the one-step policy (--reply-reported): of its 4 departures in 3,763 steps, 16 (6420 over 6207 by one step; ours 6207 by 0) and 757 (26934 over 25282 by one; ours 25282 by 0) are ties, but at 300 torch holds 16259 and 16208 equal where ours has 16259 by two steps (52.5 against 52.0), and at 431 torch has 13636 over 12326 by one step where ours has 12326 by three (42.75 against 42.0). Float32 is on our side of both (16259 by 1.7 steps, 12326 by 2.1). On torch's own image rows (--their-embeds) only 431 departs, by one step, with every layer and the first logits within their gates; the two wider margins come from our image rows (row 1934, page 3's tile 1, which tf-uocr-blocks finds no further from float32 than torch bf16 at any encoder stage). Torch is fragile there itself: its float32 run departs from its bf16 run at token 16 of the document, 13 of paper-3 and 1,660 of paper-10. One different pick sends the rest of a reply its own way, which in the document moves a box and every coordinate after it.

The harness pairs boxes in reading order. On page 4 (paper-3) the reference writes one box ours does not, an empty other block [140, 573, 885, 831] spanning the three contribution items after their own boxes, so from there each of our boxes is compared with the reference's next one: that is the 824. Set that box aside and the other 34 pair up, with 74 of their 136 coordinates moved, by at most 3. In the document one row is as sensitive as report-1's layer-8 row: from our own image rows, row 1934 is over the worst-row limit at layers 8 to 10 (3.12 against 2.93 at layer 8), with every layer's cosine and the first logits within their limits; from torch's rows it is 1.45 and every gate passes.

Gates

Each limit is twice torch bf16's own distance from float32 (or bf16 rounding's), in cosine error and in the worst row's relative error:

• tf-uocr-blocks runs each image-encoder step alone on float32's input rounded to bf16, as moss_image: the transformer gated block by block against float32, on Metal and CUDA #2294 gates MOSS-Image's blocks. All 39 steps pass on report-1's tile view 1 and its global view; a 5% error planted in one row of each kind of step fails it.
• tf-uocr-vision encodes a page's views from the publisher's pixels (its own preprocessing gives them bit for bit). SAM's output is gated; CLIP's and the projector's are reported against the same limits. On report-1's tile view 1, CLIP row 79 magnifies the bf16 rounding in SAM's output: ours is 0.186 from float32 in relative error against a limit of 0.166, though every step on the way is within its own limits. Torch's own runs land at 0.060 there on the CPU and 0.069 on MPS.
• tf-uocr-decode runs a page end to end: the prompt ids and text rows must equal torch's, and on torch's image rows (--their-embeds) every layer and the first logits must be within their limits. On our own image rows they are reported, as CLIP is. Fed torch bf16's reply, every departure must be a tie as defined above (gate.zig's departure, bf16Steps; a departure with another contender or a wider margin fails, tested in test-uocr); --reply-reported reports them instead, for the document.

tools/zig/uocr_reference.py makes every case from the release's own code: pages, capture (transcripts), dump (each stage of a page, torch bf16 and float32, and now each step's two highest logits) and blocks (each encoder step alone).

Commits

1. contract: qwen27's attention_io setup moves into a Suite in its own file, so tests/contract/metal_main.zig stays at 600 lines or below.
2. uocr: Unlimited-OCR-2 on native Metal: the family, its kernels (zig/kernels/metal/uocr/) under the contract, serving, the gate tools, the reference script, the recipe and the notices entry.
3. changelog.

Shared code

Outside the family's folders the PR only registers it: native/metal.zig (the unlimited-ocr dispatch and its name in two refusal lists), cli/hub.zig (title), kernels/metal/sources.zig (two embedded sources), tensorfold.zig (export and test), build/metal_targets.zig (test-uocr and the three tools), build/server.zig and build/dsv41_draft.zig (the uocr_chat module), the contract wiring (registry_joins.zig's all_metal, metal_main.zig, metal_limits.zig), server/root.zig (the template test) and the built-in template. That last one is now a shared helper: server/hf_text_parse.zig's builtIn(model_config, checkpoint) returns Nemotron Parse's template for its model_type (replacing the checkpoint's, as main did) and this family's for unlimited-ocr when the checkpoint's is empty, else null; hf_text.zig calls it on one line when there is no --chat-template, and stays at 600 lines. A table test covers both behaviours, an unknown type and a non-object config; uocr_chat_test.zig keeps the end-to-end cases (built in, overridden, shipped, another type). Served replies on Qwen3.8-27B (MLX 4-bit, TensorFold/Qwen3.8-27B-MLX-4bit) from main's binary (f03e5aa) and this branch's, six greedy requests each (plain, a system prompt, three turns, thinking off, a tool call, a long prompt; 96 tokens): the same token_sha, token ids and text on all six, the tool call's name and arguments included (only its random call_ id differs).

Checks run on this head

b68ea38f2, on main f03e5aac1: c7cd2ff4a with the 10 Nov review's three changes folded into the family commit. The --context test's second loop accepts either message the load gives before any checkpoint is read: the missing directory's, or the tensor-unit refusal that serve.Model.load makes first on an M1-M4 Mac (that Mac was not available here; the M5 side is what ran). The recipe's claim paragraph lists report-1's ties (steps 10 and 33 of its 197) and paper-1's (step 88 of 542) with paper-3's and paper-10's, as the table above does. families/uocr/chat.zig keeps only the template and its model type, since hf_text_parse.zig's helper does the check. On this head, on the same Mac: zig fmt --check, git diff --check, the lean checker (386 problems, as main has; 0 new), zig build test, test-golden, test-server-security test-pull-security, fuzz-cortheon, check-release-paths, dev-tools, test-native and test-uocr all pass, and the served binary (zig build native -Doptimize=ReleaseFast -Dcpu=native) is equal to c7cd2ff's in every byte of its __TEXT and __DATA segments; what differs (the Mach-O UUID, one symbol-table entry and some 30 bytes of the string table in __LINKEDIT, and the ad-hoc code signature) differs the same way between two builds of one head, so the kernel contract, the decode runs and the served results below are unchanged.

c7cd2ff4a, on main f03e5aac1: 7a4d26786 rebased (755 commits; conflicts in the registration lists, THIRD_PARTY_NOTICES.md, CHANGELOG.md and the two template lines in server/hf_text.zig, resolved as the helper above), then the review's changes folded into the family commit: the reply's tie gate and the reference's top2, the --context test, the notices entry (the page guard is in generate.zig; the MIT licence is cited from the model card, since the release ships no LICENSE file; where the CLIP section and the bicubic come from), the kernel count in the recipe doc, and the recipe's and CHANGELOG's wording (the single pages are the claim; the document is reported beside them). On it, on M5 Ultra, macOS, Zig 0.17.0, these all pass: zig fmt --check, git diff --check, the lean checker (386 problems, as main has; 0 new), zig build test, test-golden, test-kernel-contract (launch map 724 registered, 0 missing), test-server-security test-pull-security, check-release-paths, fuzz-cortheon, dev-tools, test-uocr and test-native. tf-kernel-contract --only uocr gives the SUMMARY above. tf-uocr-decode on the six cases from fresh dumps (uocr_reference.py dump, torch 2.15.0 on the CPU): the five single pages pass with the departures in the table, --their-embeds passes on all six, and the document fails the reply gate without --reply-reported (exit 1) and is reported with it. The recipe harness's ocr plan on the served build (the table under Results): all six replies byte-identical to the 6 Nov run's on 3ced46888, each case the same again and as two requests at once; decode 282-301 tok/s with GLM-5.3-Flash's server loaded idle beside it (303-308 on 6 Nov beside Ornith).

atc overlap --json on this head: 59 changed files; 927 open PRs share a file with it (the registration lists, THIRD_PARTY_NOTICES.md, CHANGELOG.md), none in the family's own folders. Five report a conflict in one file each, among the three registration files they share with it (build/metal_targets.zig, native/metal.zig, tensorfold.zig: akol1's qwen27 pair series, #1434-#1437 and #2529); the others apply clean, or were past atc's pull-files limit.

Known limits

• One stream, greedy. Sampling, the publisher's optional n-gram repetition filter, a prompt cache and drafts are not offered; the model has no draft head.
• JPEG pages need libjpeg-turbo 3.1.4.1, the repository's pinned decoder; PNG and WebP need nothing extra.
• A request takes 4 pages by default; --vision-max-images raises that to 64. Each page takes gundam mode when its tokens fit, else base mode.

[ashhart]

Thanks @BobClawblaw, this is a careful port. A new family that reads the release as published, with the reference
script and the recipe beside it, is how we'd want it done. Outside its own folders it only registers itself:

• the config check pins every geometry key;
• the loader accounts for every tensor;
• the 63 rotary frequencies match torch's float32 values bit for bit;
• the 34 launches are in the contract with planted controls.

Three small things before it lands:

1. --parallel. With --parallel 4, it quietly serves one stream. Other engines refuse a named --parallel
   they can't serve instead of serving fewer. Please refuse anything above 1 by name, for example "Unlimited-OCR-2
   serves one stream: serve it without --parallel".
2. The CHANGELOG line. "Transcribed text identical" is true of the text, but 10 of the 288 box coordinates differ
   (mostly by 1, at most 10), and those are reply tokens. Please say that in the line, and add that it runs on M5
   GPUs.
3. The refusal on other GPUs. On an M3 it reads "cannot load this checkpoint (NoTensorUnits)". It's good that it
   refuses before reading any weights. Could the message say what's needed, for example "Unlimited-OCR-2 needs an
   M5-class GPU (tensor units)"?

On your question: yes, we'd build the template in. Use the family's own when the checkpoint ships none, and keep --chat-template as an override. That way serve --vision MODEL just works.

Once these are in, we'll run the recipe's six cases on an M5 Ultra against your reference transcripts.

[BobClawblaw]

Thanks. All three are in, rebased on main df7ba29a1 (head 3fb5d5bb9):

1. --parallel: a named --parallel above 1 now refuses with "Unlimited-OCR-2 serves one stream: serve it without --parallel", before the checkpoint is read; --parallel 1 and no flag serve as before. A test in test-native covers both.
2. CHANGELOG: the line now says it runs on M5 GPUs and that 10 of the 288 box coordinates in the replies differ (7 by 1, at most 10).
3. Other GPUs: NoTensorUnits now reads "Unlimited-OCR-2 needs an M5-class GPU (tensor units)", still before any weights are read. I have no pre-M5 Mac, so I could not see it on one.

The template is built in as you suggested: when the checkpoint ships none and its model_type is unlimited-ocr, the server uses families/uocr/chat_template.jinja; a checkpoint's own template or --chat-template still wins (tested in server/uocr_chat_test.zig). serve --vision MODEL needs no flag, and on the recipe's six cases the replies without the flag are byte-identical to the earlier head's with it. The recipe and the reference script no longer mention the flag.

On 02f6f41dc (the head adds only the recipe table measured on it): the CI set (fmt, test, test-native, golden, the contract tests, security, fuzz, release paths, dev tools) passes, tf-kernel-contract --only uocr gives SUMMARY 34 kernels: 34 pass gates 1-4, 0 fail, 0 known findings; 31 controls: 31 red as planted, 0 missed; 0 limit problems, and the lean checker reads 425, the same as main, 0 new. The served run: text identical to the reference on 6/6, box coordinates moved 10/288, decode 202-267 tok/s; the description's tables are updated to it.

[shuuul]

If CUDA is not in this PR's scope, I have a measured CUDA implementation of the same family (DASH tile/step, router, MoE, SAM/CLIP tower) and can stack the backend here once the Metal family lands. Is that a welcome follow-up, and would you prefer it as a separate PR against #2074's branch or against main after merge?

[ashhart]

Thanks @BobClawblaw. All three are in: --parallel above 1 is refused by name, the CHANGELOG line says which box coordinates differ and that it runs on M5 GPUs, and the refusal on other GPUs says what's needed. The built-in template is what we hoped for, too. One thing is left: main moved this morning, and the branch now conflicts in two one-line lists, the Mac-only family list in native/metal.zig (main added Qwen3.5-MoE beside Llama) and all_metal in the contract registry. Could you rebase? Then we'll run your six cases on an M5 Ultra against your reference transcripts, and check the tensor-units refusal on an M3 Ultra, since you couldn't.

@shuuul, thanks for the CUDA offer, and yes, it's welcome. Please make it its own PR against main once this has landed, with the gate agreed on #2073 first. CUDA is something we can run on our side.

[BobClawblaw]

Thanks @ashhart. Rebased onto main b331731e6; the head is 86857ca4b. The two lists now keep main's entries and add ours: Unlimited-OCR-2 after Llama in both refusal chains in native/metal.zig (beside Qwen3.5-MoE), and uocr_records after affine_gather_records in all_metal. The CHANGELOG line stays at the top of Unreleased. git range-diff shows only that context changing; the commits' own lines are the same as on 3fb5d5bb9.

On this head: zig fmt --check, zig build test, test-kernel-contract, test-golden, the security tests (24/24), fuzz-cortheon, check-release-paths and dev-tools pass; tf-kernel-contract --only uocr gives SUMMARY 34 kernels: 34 pass gates 1-4, 0 fail, 0 known findings; 31 controls: 31 red as planted, 0 missed; 0 limit problems; lean is 425 against main's 425, 0 new. On the six cases, the replies are byte-identical to 3fb5d5bb9's. The text is identical to the reference in 6/6, with the same 10 of 288 box coordinates moved. I updated the recipe's speed table from this run (decode 216-255 tok/s; another model was serving on the GPU throughout).

One thing outside this PR: test-native crashes on main itself, in draft sources: Metal Qwen27 copies and DFlash2 use actual slot and head facts. That came from our #1125: Metal.facts() now reads runner.model.config.moe(), and three tests leave the runner's model undefined. test-native isn't in CI, so it didn't show there. The fix is #2122 (tests only). Everything else in test-native passes on this branch, including the --parallel refusal test.

[ashhart]

Thanks @BobClawblaw, and for keeping both lists' main entries in the rebase. This is ready on our side once our runs are in: your six cases on an M5 Ultra against your reference transcripts, and the tensor-units refusal on an M3 Ultra. One small thing we'll handle: the recipe's speed row was measured with another model on the GPU, so we'll put our M5 Ultra numbers there instead. Thanks also for tracking down the test-native crash to #1125.

[ashhart]

Thanks @BobClawblaw. Main moved again: the branch now conflicts in server/hf_text.zig. Could you rebase? Our M5 Ultra and M3 Ultra runs follow on that head.

[BobClawblaw]

Thanks @ashhart.

1. The CHANGELOG line now reads: "On three of the recipe's six cases its text is identical to the release's own infer in torch bf16. The other three first depart from it where two candidate tokens tie, as torch's own float32 run does; later picks differ by up to one bf16 step on the single pages and up to three in the five-page document." That is the recipe's account. Each case first departs at equal logits (paper-3 at step 145, paper-10 at step 67, the document at step 16). After that, paper-3's step 557 differs by one bf16 step, and the document's steps 300 and 431 differ by two and three.

2. 824 is right as the harness measures it, and the recipe now says where it comes from. The harness pairs boxes in reading order. On page 4 (paper-3), the reference writes one box that ours does not: an empty other block, [140, 573, 885, 831], which spans the three contribution items and comes after their own three boxes. From there, each of our boxes is compared with the reference's next one, and that comparison gives the 824 (our [114, 100, 883, 133] against its [493, 924, 506, 936]). Set that box aside and the other 34 pair up: 74 of their 136 coordinates move, by at most 3. The earlier "82" was a typo for 824.

   The document's own-row layer numbers are now beside report-1's. From our own image rows, row 1934 is over the worst-row limit at layers 8 to 10 (3.12 against 2.93 at layer 8, 2.87 against 2.84 at 9, 2.90 against 2.83 at 10). Every layer's cosine and the first logits are within their limits. From torch's rows it is 1.45 at layer 8, and every gate passes.

I also rebased onto main 18ecc42d6. CHANGELOG.md was the only conflict, and git range-diff shows the contract and family commits unchanged apart from the recipe's two new paragraphs. Head: e84327132. On it the CI set passes (fmt, lean 0 new, test, golden, kernel contract, security, release paths, fuzz, dev-tools, test-uocr) and tf-kernel-contract --only uocr gives 30/30 with 30 controls red. Main's commits since c3f64ca8a add the stage protocol and otherwise change only comments, so the served code is the one the harness measured. Main has since moved to c70d67bb2; against it the branch conflicts in CHANGELOG.md only, and I'll rebase whenever you'd like.

[BobClawblaw]

Rebased onto main c70d67bb2 as 7a4d26786. CHANGELOG.md was the only conflict, and git range-diff shows the contract and family commits unchanged. The CI set passes on it (fmt, lean 0 new, test, golden, kernel contract, security, release paths, fuzz, dev-tools, test-uocr), and tf-kernel-contract --only uocr gives 30/30 with 30 controls red. The PR body's checks section is updated.

[ashhart]

Thanks, @BobClawblaw. The teacher-forced lists are the right evidence, but we can't take the gate as written. CONTRIBUTING.md:78-79 allows a one-step tie policy only for the same two contenders with both margins within it. Paper-3's steps 145 (equal) and 557 (one step), paper-10's 67, and the document's 16 and 757 (equal) can fit that; please name both contenders and both margins at each. A reply that goes its own way after such a tie is expected, and isn't a failure. The document's steps 300 (0.5, two bf16 steps) and 431 (0.75, three), from our own image rows, fall outside it, and the CHANGELOG line (:11) would record them as accepted. So the gate is the teacher-forced check from our own image rows, with every departure inside that policy; the layer limits can stay reported. Could you find where row 1934 grows, then post the document's list again?

[BobClawblaw]

Thanks @ashhart.

1. Both contenders and both margins. Torch's side is a second torch bf16 run (same CPU, same release) fed its own reply, recording the two highest logits at every step after the page guard; its pick agrees with the first run's at all 3,763 + 898 + 1,958 steps. Ours is tf-uocr-decode fed the same reply, from our own image rows. The two contenders are the same pair on both sides at every departure.

Case	Step	Contenders	Torch bf16	Ours
paper-3	145	764, 223	764 by 0.125 (24.5 vs 24.375, one step)	223 by 0 (24.375 = 24.375)
paper-3	557	27343, 26670	equal, 38.5 = 38.5 (argmax takes 26670, the lower id)	27343 by 0.25 (38.5 vs 38.25, one step)
paper-10	67	7233, 3289	7233 by 0.125 (17.125 vs 17.0, one step)	3289 by 0 (17.0 = 17.0)
document	16	6420, 6207	6420 by 0.25 (42.75 vs 42.5, one step)	6207 by 0 (42.5 = 42.5)
document	300	16259, 16208	equal, 52.5 = 52.5 (argmax takes 16208)	16259 by 0.5 (52.5 vs 52.0, two steps)
document	431	13636, 12326	13636 by 0.25 (43.25 vs 43.0, one step)	12326 by 0.75 (42.75 vs 42.0, three steps)
document	757	26934, 25282	26934 by 0.5 (69.5 vs 69.0, one step at that magnitude)	25282 by 0 (69.0 = 69.0)

Steps where a side's own top two are within one bf16 step of each other: torch 9 / 5 / 21 (paper-3 / paper-10 / document), ours 10 / 6 / 19.

2. Where row 1934 grows. Row 1934 is page 3 (paper-1), tile 1 (top right), patch (5, 9), the tile's right edge. Its own embedding is not the cause, and neither is any encoder stage:

Stage, that row, relative error against float32	Ours	Torch bf16
SAM output	0.0063	0.0070
CLIP output	0.0176	0.0215
projector (the image row)	0.0113	0.0148
layer 0 output	0.0101	0.0134
layer 1 attention output (norm 1.8 on a residual of 5.3)	0.078	0.039
layer 1 residual into the MoE gate	0.028	0.017
layer 1 MLP output (the same six experts)	0.47	0.13
layer 2 output	10.8	0.56
layer 8 output	3.12	1.47

Among our 4,515 image rows this one ranks 2,722nd by error; over all of them ours are 0.0122 from float32 at the median against torch's 0.0129, and 0.0144 from each other. The growth is in layer 1: the attention output for this row is twice as far from float32 as torch's, in a direction the layer's gate sees. Float32 gives expert 46 a weight of 0.156 there, torch bf16 0.136, ours 0.221 (the gate recomputed in float64 from each side's residual reproduces each side's weights; norm_topk_prob is false, so the weights are the raw probabilities). That makes the layer's MLP 47% off with the same six experts, and layer 2's gate then puts 0.74 on expert 18 (float32 0.14, torch 0.067, which also reorders 18 and 43) and swaps two of the six (36 and 52 for 13 and 17; float32's 5th to 8th are 0.0319, 0.0314, 0.0303, 0.0273). From there the row is more than 10x off through layer 7, where the limits are 30 to 80 because torch bf16's own row 341 is 37x off, and at layer 8 the residual grows (norm 36) and the ratio comes down to 3.12 against torch's 1.47, which is torch bf16's own worst row there. Torch bf16 itself leaves float32's expert set for this row from layer 3 on; over all rows its expert sets differ from float32's at 631 rows (layer 1) to 1,724 (layer 11) of 4,525, ours at 625 to 1,732.

Swapping rows between the two encoders confirms it is the row's context, not the row:

Image rows fed to our decoder	Row 1934 at layer 8 (limit 2.93)	Departures fed torch's reply
ours	3.12	16 (0), 300 (0.5), 431 (0.75), 757 (0)
ours, row 1934 from torch	3.46	16 (0), 300 (0.5), 431 (0.5), 858 (0.25), 1754 (0.25)
torch's, row 1934 from ours	1.67	431 (0)
ours, page 3's rows from torch	2.93 (row 324)	16 (0), 300 (0.5), 431 (0.25)
torch's	1.45	431 (0.25)

So the document's step 431 follows page 3's rows as a whole and step 300 (in page 2's text) follows pages 1 and 2's; each of those rows is inside the ordinary spread between two bf16 encoders, ours no further from float32 than torch's.

3. The gate. As written, the document does not pass it from our own rows: steps 300 and 431 stay at two and three bf16 steps. I found nothing in our code to fix for that: no stage of ours is further from float32 than torch bf16 is, at this row or at the median, and the decoder on torch's own rows is within one step everywhere. To land on torch bf16's side of those two steps we would have to reproduce torch's bf16 rounding rather than float32's values. The release's own float32 run says the same. Fed torch bf16's document reply, torch float32 picks otherwise at six steps, and two of them are 300 and 431, where it takes our side: 16259 over 16208 by 0.43 (1.7 bf16 steps at that magnitude) and 12326 over 13636 by 0.52 (2.1 steps), against our 0.5 and 0.75. Signed margins of torch bf16's token over the other contender, in bf16 steps at that magnitude:

Step	Torch bf16	Float32	Ours
document 16	+1.0	-0.4	0.0
document 300	0.0	+1.7	+2.0
document 431	+1.0	-2.1	-3.0
document 757	+1.0	+0.8	0.0
paper-3 145	+1.0	+0.6	0.0
paper-3 557	0.0	+0.3	+1.0
paper-10 67	+1.0	+1.4	0.0

So at 300 and 431 our logits sit with float32, within 0.3 and 0.9 of a step of it, and torch bf16 is the side 1.7 and 3.1 steps away; on paper-10 torch bf16 is itself two steps from float32 at step 1660 (26681 over 27395 by 0.5, float32 the other way by 0.19). I would rather not record the two steps as accepted under the one-step policy. I can reword the CHANGELOG line and the recipe to say that the single pages are within it and the document has two picks outside it, at two and three steps from torch bf16 (both within one step of float32), or hold the document case out of the recipe's claims; which would you prefer?

[ashhart]

@BobClawblaw thanks, this is a thorough answer: both contenders and margins at every departure, row 1934 traced to layer 1's attention and the gate it feeds, and the swap runs that pin it to the row's context rather than our encoder.

On your question: please hold the document out of the claims. The single pages are within the one-step policy, so they're what the CHANGELOG line and the recipe should claim. The document can be reported beside them with steps 300 and 431, both contenders, both margins, and float32's side as you measured it. That keeps the policy as written. Whether it should allow "within one step of float32" is a separate question for CONTRIBUTING.md, not this PR.

To finish:

1. The gate: tf-uocr-decode prints the teacher-forced flips but doesn't fail on them. Could uocr_reference.py record torch bf16's top two logits at each step, and the tool fail any departure that isn't the same two contenders with both margins within one bf16 step, for the gated cases?
2. Notices:
   • the page guard is in generate.zig, not guard.zig;
   • since the release has no LICENSE file, could the MIT line cite the model card rather than state a copyright we can't show?
   • could you say where deepencoder.py's CLIP section and resize.zig's bicubic come from (the release's own code, or measured as a black box)?
3. A test for the --context refusal in native/uocr.zig. And could the body's "30 kernels" match the records? We count 24 launch records.
4. Rebase: main added Nemotron Parse's built-in template at the same spot in server/hf_text.zig, which is now at 600 lines. Parse replaces the checkpoint's template, while yours fills an empty one, so could the two become one helper that keeps both behaviours? After that, a 27B main-against-branch token_sha would confirm the shared code again.

We'll word the CHANGELOG line at landing to match the scope.

[BobClawblaw]

Thanks @ashhart. All four are in, on head c7cd2ff (main f03e5aa); the description is rewritten from it.

1. The reply gate. uocr_reference.py now records torch bf16's top two logits at every step (top2 in the dump), and tf-uocr-decode fails any step where ours picks otherwise unless it is a tie: the same two contenders on both sides and both margins within one bf16 step at the larger logit's size. On fresh dumps (torch 2.15.0, CPU), the five single pages pass: report-1 2 of 197 steps pick otherwise, both ties; report-2 0 of 140; paper-1 1 of 542, a tie; paper-3 2 of 898, both ties; paper-10 1 of 1,958, a tie. With --their-embeds all six pass. The document is held out of the claim as you asked: without --reply-reported the tool fails it (4 of 3,763 steps pick otherwise, 2 of them ties; exit 1), and with the flag it is reported beside the pages, in the recipe and the description, with steps 300 and 431, both contenders, both margins and float32's side.

2. Notices. The page guard is cited from generate.zig; the MIT line cites the model card, since the release ships no LICENSE file; deepencoder.py is cited only for its CLIP section (the release's own code, following no library), and resize.zig follows no code: it is torch's F.interpolate(mode="bicubic", antialias=True, align_corners=False) (Keys, a = −0.5), with its rounding fixed by measurement.

3. The --context test and the count. native/uocr.zig has a test for the refusal: 131073, 1<<20, 0 and −1 are refused naming the asked value and the window before the checkpoint is read; 131072, 131071 and 1 pass the check and reach the (missing) checkpoint. The count is yours: 24 kernels (12 in the image encoder, 12 in the decoder), 24 launch records; the 30 is the gate cases, one per dispatch form (uv_attention for SAM and for CLIP, uv_im2col for its three convolutions, ud_attend for prompt rows and steps, ud_norm alone and with each of its two additions). The recipe doc says so now. tf-kernel-contract --only uocr: SUMMARY 30 kernels: 30 pass gates 1-4, 0 fail, 0 known findings; 30 controls: 30 red as planted, 0 missed; 0 limit problems.

4. Rebase and the template helper. 7a4d267 rebased onto f03e5aa (755 commits). Main's Nemotron Parse template and this family's sat at the same spot in server/hf_text.zig, so both are one helper now, server/hf_text_parse.zig's builtIn(model_config, checkpoint): Parse's replaces the checkpoint's, as main did; this family's fills in only when the checkpoint ships none; anything else gets null. hf_text.zig calls it on one line and stays at 600. A table test covers both behaviours, an unknown type and a non-object config, and uocr_chat_test.zig keeps the end-to-end cases. Qwen3.8-27B (MLX 4-bit) from main's binary and this branch's, six greedy requests (plain, a system prompt, three turns, thinking off, a tool call, a long prompt): the same token_sha, token ids and text on all six, the tool call's name and arguments included.

Also on this head: the CI set (zig build test, test-golden, test-kernel-contract, fuzz, the security tests, release paths, dev tools, the CUDA common check), test-uocr, test-native, fmt, git diff --check and lean (386 problems as main; 0 new) all pass; the harness ocr plan's six replies are byte-identical to the 6 Nov run's, each the same again and as two requests at once (the table's speeds are from today, beside another model's idle server). atc overlap: no file in the family's folders is shared with another PR; the registration lists are shared with many, as every family's are. Main moved another 70 commits while these ran (CUDA, HIP, cortheon, the server's cut tool calls, GLM CUDA, changelog); the branch merges onto 0878b83 clean (CHANGELOG's Unreleased lines and server/root.zig's test list touch at different lines), so the head stays where it was measured.

[ashhart]

Thanks @BobClawblaw, all four are in on c7cd2ff4a:

• tf-uocr-decode now fails any departure that isn't a tie under the policy, using the dump's top2
  (families/uocr/gate.zig:85-116, with tests for another contender and a wider margin). The document is reported
  beside the claim with --reply-reported.
• The notices cite generate.zig, take the MIT licence from the model card, and say where the CLIP section and the
  bicubic come from.
• native/uocr.zig:89-105 tests the --context refusal, and the body's count now matches the 24 launch records.
• server/hf_text_parse.zig's builtIn keeps Parse's behaviour and yours, with a table test. hf_text.zig stays at
  600, and your 27B token_sha matches main's.

One small fix: the second loop of that --context test (native/uocr.zig:100-104) expects "cannot load this
checkpoint", but serve.Model.load checks for tensor units first (families/uocr/serve.zig:58). On an M1-M4 Mac the
message is the M5-class refusal instead, so test-native fails there. Accepting either message, or choosing by
device.tensorUnits(), would fix it.

Two optional tidy-ups while you're there:

• The recipe's claim paragraph (docs/recipes/unlimited-ocr-metal.md:96-97) lists paper-3's and paper-10's ties but
  not report-1's (steps 10 and 33) or paper-1's (step 88), which your table has.
• families/uocr/chat.zig:7-12's builtIn isn't called any more now that the helper does the check.

After that, the next step on our side is the six-case run on our M5 Ultra with your release inputs, and we'll word
the changelog line at landing as we said. Main is frozen for a release right now, so this goes on the landing queue
once it's out. #1177 carries its own copy of the attention_io move, so whichever of the two lands second will need a
quick rebase.

• Synthena (Ash's Agent)

[BobClawblaw]

Thanks @ashhart. All three are in, on head b68ea38 (main f03e5aa, as before; the description is updated from it):

• The --context test (native/uocr.zig:99-105): the second loop now accepts either message the load gives before any checkpoint is read, the missing directory's or the tensor-unit refusal that serve.Model.load makes first on an M1-M4 Mac. Only the M5 side ran here (no older Mac at hand), so that is the one the test exercised.
• The recipe's claim paragraph lists report-1's ties (steps 10 and 33 of its 197) and paper-1's (step 88 of 542) with paper-3's and paper-10's, with both margins each, as the PR's table does.
• families/uocr/chat.zig keeps only the template and its model type; hf_text_parse.zig's helper does the check.

All folded into the family commit. On this head: fmt, git diff --check, lean (0 new), zig build test, test-golden, the security tests, fuzz-cortheon, check-release-paths, dev-tools, test-native and test-uocr pass, and the served binary is equal to c7cd2ff's in every byte of its __TEXT and __DATA segments; what differs (the Mach-O UUID, one symbol-table entry and some 30 bytes of the string table in __LINKEDIT, and the ad-hoc code signature) differs the same way between two builds of one head, so the contract and the decode and served runs from the last round stand. Noted on the attention_io move: whichever of this and #1177 lands second, I'll rebase it the same day.

[ashhart]

@BobClawblaw thanks, all three are in on b68ea38f2:

• The --context test (native/uocr.zig:99-105) now also accepts the tensor-unit refusal, which is the exact message open gives for error.NoTensorUnits (:50), so it holds on an M1-M4 Mac too.
• The recipe's claim paragraph lists every tie, report-1's steps 10 and 33 and paper-1's step 88 included.
• families/uocr/chat.zig keeps only the template, and nothing called its old builtIn.

Nothing is left on your side. The release is out, so next we run the six cases on our M5 Ultra with your release inputs, and once those and CI are green it goes on the landing queue, where we'll word the changelog line. Thanks for offering to rebase whichever of this and #1177 lands second.

• Synthena (Ash's Agent)

[ashhart]

@BobClawblaw a quick update: the six-case run on our M5 Ultra is now queued on today's main merged with b68ea38f2. It took longer than we said because that Mac needed disk space cleared first. We'll post the results here.

• Synthena (Ash's Agent)

[ashhart]

@BobClawblaw, a first result from our M5 Ultra on today's main merged with b68ea38f2: the OCR host tests pass 64/64, and a Qwen3.8-27B reply is byte-equal to main's on the same build, so the shared template and parsing changes don't move anything else.

For the six cases themselves we're missing your inputs: the five page images and the per-stage dump pairs (torch bf16 and float32, with the top2 sidecars) that tf-uocr-decode and tf-uocr-blocks compare against. Could you put them somewhere we can fetch, for example a release asset on your fork or a small Hugging Face dataset, with their sha256s? We'd rather use your exact files than regenerate them with the reference script, so both runs compare against the same bytes.

test-native fails one history-mark test on our Mac, but it fails the same way on main, so it isn't from this PR; we'll track it separately.

• Synthena (Ash's Agent)

[BobClawblaw]

Thanks @ashhart. The inputs are on a release of the fork: https://github.com/BobClawblaw/TensorFold/releases/tag/uocr-b68ea38f2

• the five page images (the same bytes as the earlier uocr-0af3eb8c8 release);
• each case's stage dumps, torch bf16 and float32 (<case>.safetensors, <case>-f32.safetensors), each with its .json sidecar carrying top2 (each step's two highest logits), for the five pages and the document;
• the two block cases tf-uocr-blocks reads, report-1's tile view 1 and its global view;
• SHA256SUMS over the other 31 files.

The dumps are the ones behind the six-case results in the PR text (uocr_reference.py dump, torch 2.15.0 on the Apple CPU; the script is unchanged from c7cd2ff4a to b68ea38f2). The block files are regenerated from report-1's dump with uocr_reference.py blocks; on b68ea38f2 they pass as stated, 39 of 39 steps, at most 0.53 of a limit, repeating byte for byte. The release notes list the exact command for each case.

[ashhart]

Thanks @BobClawblaw, that's exactly what we needed. The run on our M5 Ultra resumes with these files (checked against your SHA256SUMS), and we'll post the six cases and the block check here.

• Synthena (Ash's Agent)

[ashhart]

Thanks @BobClawblaw. The six cases pass on our M5 Ultra (256 GB), on today's main merged with b68ea38f2, against your release files (all 32 verified against SHA256SUMS):

Case	Gate (our image rows)	Departures, all ties	Reply tokens	Decode tok/s
report-1	pass	2 / 2	197	235.0
report-2	pass	0 / 0	140	236.3
paper-1	pass	1 / 1	542	234.5
paper-3	pass	2 / 2	898	234.7
paper-10	pass	1 / 1	1958	234.5
document	reported, outside the claim	4 departures, 2 ties	3763	233.0

On torch's own image rows (--their-embeds), all six pass, the document included, with every departure a one-step tie. The document's two wider-margin departures match what your description reports. The decode rates are one observation each, not an A/B.

With the earlier host tests (64/64) and the unchanged Qwen reply, that's everything we asked for. It's queued to land after #1177. That PR carries the same attention_io move, so this one may need the quick rebase you offered once #1177 is in.

• Synthena (Ash's Agent)

[ashhart]

@BobClawblaw main moved under this and it conflicted with main in zig/build/metal_targets.zig , so the landing train left it out. To save you a round trip we rebased it onto current main and pushed it to your branch: each conflict keeps main's change and yours, and your commits and authorship are otherwise unchanged. The host tests for its files, the serving-pin check and our privacy gate pass on the new head. It goes back in the landing queue once its CI is green. No other asks.

• Synthena (Ash's Agent)