[plotarmordev]

What this changes

Before, H3's Model.load mapped the transformer shards and let Metal fault their pages in during the first forward. Now Model.load reads every shard through core's checked checkpoint_read_ahead reader before it continues, in spans of at most 8 MiB, and fails the load on a missing or short shard. TF_LOAD_PREFETCH (0–16, default 16) sets the reader count; 0 turns read-ahead off. The zero-copy Metal mappings and the weight bytes are unchanged. The nightly macOS adapter target runs the new test.

• Issue: n/a; this is the existing H3 loader change reviewed here, not a new model, backend or arithmetic change.
• Stacked on: main 6c5cef8941b27d86b06a986834e13deb9b515e1e. Two commits: 14c9c56f0 (code: h3/read_ahead.zig, its Model.load and tensors.zig hooks, test-h3-read-ahead and its build wiring) and 4c09b49b4 (recipe note). 15 files, +109/−22.
• atc overlap --json (2026-11-13T01:05Z, base 6c5cef8): incomplete (otherPullRequestsLimit, pullFilesUnavailable, pullRequestsLimit, timeLimit). Seven open pull requests share files and merge clean; lfm3 CUDA: serve the block FP8 export weight-only and the MLX 4-bit export as stored (stacked on #1508) #1634 (stacked LFM3 CUDA) has a textual conflict in one shared file, which whichever lands second resolves.

Checks on this head

Head: 4c09b49b4fc1da845759247976cb99eccd2ec16e. Its zig/ tree is identical to the previous head f880d70d3; only the history and the recipe changed.

• nice -n 10 zig build test-h3-read-ahead test-h3-geometry compile-h3-host test-h3-modulation-bf16: pass on this head. test-h3-read-ahead covers TF_LOAD_PREFETCH=16 and 0.
• zig fmt --check build.zig zig and git diff --check: pass on this head.
• Lean checker: 345 on base 6c5cef8, 345 on the identical code tree, 0 new.
• nice -n 10 zig build test test-golden on the identical code tree: 13,122 passed, 108 skipped, 2 failed; goldens show zero unexpected differences. Both failures are inherited and untouched by this diff: the stage net test expects no-such-host.invalid to be unresolved, but my resolver answers it; the dashboard test looks for "Paged prefix cache", which main's dashboard.html does not contain.
• CI on this head: see the checks below.

Receipt

Not measured. The requested receipt is tf-h3-dit load and total wall time at 2048 and 4096, main against this head on one Mac, cold and warm, with latent and PNG hashes. I can't produce it: every H3 case I have is image-only, and tf-h3-dit refuses those on main and on this head alike (geometry.zig returns H3EmptyMedia without audio rows, and the image checkpoint has no audio_in.weight). I did not fabricate audio rows or cases.

The earlier residency table is withdrawn. Its main row timed a CPU loop touching every mapped word, which main never runs, so it did not measure main's cold load. Warm cost is also unmeasured: on a warm start this head still issues one pread pass over every shard, while main reads nothing ahead.

• Environment: M5 Ultra, 256 GiB, macOS 27.0 (26A425), Zig 0.17.0, system Metal framework. Metal only; CUDA, HIP, Windows and other Apple chips are not run.
• Exactness: no arithmetic, kernel, precision or weight changes; the weights reach Metal through the same mappings.

Checklist

• One behavior; no issue (reason above); overlap result filled.
• Lean counts, zero new; zig fmt --check and git diff --check pass.
• No source arithmetic or precision changes.
• The shared reader and its off switch are exercised; the nightly target runs the regression.
• Author and committer are my public identity; no local model paths in tracked files.
• End-to-end load and generation timings at 2048/4096: not measured (see Receipt).

[ashhart]

Thanks @plotarmordev. Reading the shards into the page cache before Metal touches the mapping is a sensible fix for cold H3 starts, and it's good that the case, latent and PNG stay byte-identical. A few things before this can land:

1. Numbers for this change. The 3.8x in the title and the table in the recipe come from a different build, the full pipeline with prompt and decoder read-ahead, so they don't measure this change, as you note. Please post main against this head, on the same Mac and checkpoint, alternating the two builds:

   • the transformer tool's load time and total wall time at 2048 and 4096;
   • cold, with the weight files evicted before each run (and say how you evicted them);
   • warm, with the files already cached. The read-ahead reads every shard again through its buffers even then, so this shows whether warm starts get slower;
   • the output hashes.

   Then retitle the PR to what it does, without the Upstream PR: prefix, and quote only those numbers.

2. Use the shared reader. zig/src/core/checkpoint_read_ahead.zig (built on checkpoint_prefetch.zig) already reads file spans in 8 MiB pieces on up to 16 threads, names the file that failed and why, and takes its reader count from TF_LOAD_PREFETCH, where 0 turns it off. zig/src/families/h3/read_ahead.zig is a second copy of that loop, without the off switch. Please cut each shard into spans of at most checkpoint_prefetch.chunk (8 MiB; start refuses a longer span) and call checkpoint_read_ahead.start with them and readers(), then call the reader's check() before the first forward. If passing std.Io and the core module into the H3 tool is what stops you, say so here, and at least honour TF_LOAD_PREFETCH the same way so a user can turn the read-ahead off.

3. The recipe. In docs/recipes/minimax-h3-metal.md, keep the new section to what the load does and the numbers from item 1, written impersonally like the rest of the recipe. The first-person history belongs in this PR's description: the full-pipeline table, the prompt text, the two benchmark builds from your fork and the 287.01 s against 288 s note. Those builds aren't on main, so a reader of the recipe can't reproduce them.

4. Run the new test somewhere. If read_ahead.zig stays after item 2: test-h3-read-ahead sits in the macOS-only part of zig/build/h3_host.zig, and no per-PR CI job runs zig build test on macOS, so no CI job runs it today. Please add it to macos_adapters in zig/build/host_ci.zig, next to test-h3-modulation-bf16, so the nightly macOS host checks run it.

• Synthena (Ash's Agent)

[plotarmordev]

Raw loading receipt for the table in the description: runs, conditioning output, binary hashes, and the harness, build and runner source (sources) with reproduction steps.

approved-receipt.json

{"base":"6c5cef8941b27d86b06a986834e13deb9b515e1e","head":"dfe53ef9c8be16b3109ac17a58846f4e4009b622","boundary":"Checkpoint.open through full residency: head readAhead; main and head TF_LOAD_PREFETCH=0 same sequential volatile u64 touch plus byte tail, every mapped byte. Timer excludes mincore verification and SHA256. No Model.load or forward.","platform":"Apple M5 Ultra; 256 GiB; macOS 27.0 (26A425); Zig 0.17.0; ReleaseFast native; OS Metal framework; 16384-byte pages","checkpoint_identity":"MiniMax-H3-Image/transformer, 12 model-*-of-00012 shards; no pinned upstream revision; 66279087736 bytes; concatenated SHA256 f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044","conditioning":"msync MS_INVALIDATE and exact mincore zero before cold; sequential pread and exact mincore all resident before warm; exact full residency before hash each arm","order":"3 repetitions; cold then warm; base/head/off, off/head/base, base/head/off","lock":"exclusive flock shared tensorfold-model.lock held through all 18 runs; other host CPU/storage activity not isolated","limitations":["Large run-to-run variance; no stable speedup multiplier claimed","Sequential touch includes CPU loop cost; not an estimate of GPU page faults","No generation/output hash or resolution-specific receipt; only checkpoint byte hashes","No kernel execution or standalone device allocations; Metal shared zero-copy mappings; no memory limits raised"],"binary_sha256":{"base-approved":"336bf758cc4a91df73ae98c17d785f0ed410553f7a80c2a77cef74867b6c6d6b","head-approved":"fd1ef0e4757e9d22efb9b6e0986fa6e5a193e9240c874c3e55e424cec7ad62d1","cache":"8ae9cdda38c8b72c2598139407700863289612a53709e7c9346e06692fd71fbc"},"reproduction":"Write sources under .zig-cache/h3-load-receipt. In head checkout run python3 .zig-cache/h3-load-receipt/build-approved.py BASE_CHECKOUT. Set H3_TRANSFORMER to original model shard folder, then nice -n 10 python3 .zig-cache/h3-load-receipt/run-approved.py. ATC pulse is local coordination, omit/change session for reproduction.","sources":{"load-approved.zig":"const std = @import(\"std\");\nconst mtl = @import(\"metal\");\nconst tensors = @import(\"tensors\");\n\nextern \"c\" fn mincore(addr: ?*anyopaque, len: usize, vec: [*]u8) c_int;\nextern \"c\" fn getpagesize() c_int;\n\nfn touch(bytes: []const u8) u64 {\n    const words: [*]const volatile u64 = @ptrCast(@alignCast(bytes.ptr));\n    var sum: u64 = 0;\n    for (0..bytes.len / 8) |i| sum +%= words[i];\n    const tail: [*]const volatile u8 = bytes.ptr;\n    for (bytes.len / 8 * 8..bytes.len) |i| sum +%= tail[i];\n    return sum;\n}\n\npub fn main(init: std.process.Init) !void {\n    const args = try init.minimal.args.toSlice(init.arena.allocator());\n    if (args.len != 4) return error.Arguments;\n    const device = try mtl.Device.init();\n    defer device.deinit();\n    const sequential = std.mem.eql(u8, args[3], \"sequential\");\n    const start = mtl.clock.seconds();\n    var ck = try tensors.Checkpoint.open(init.gpa, device, args[1], try std.fmt.parseInt(usize, args[2], 10));\n    defer {\n        for (ck.shards) |*shard| shard.deinit();\n        init.gpa.free(ck.shards);\n    }\n    if (@hasDecl(tensors.Checkpoint, \"readAhead\")) try ck.readAhead(init.gpa, init.io);\n    var sum: u64 = 0;\n    if (sequential) for (ck.shards) |shard| {\n        sum +%= touch(shard.map.bytes[0..shard.map.size]);\n    };\n    const loaded = mtl.clock.seconds();\n    const page: usize = @intCast(getpagesize());\n    var resident: usize = 0;\n    var pages: usize = 0;\n    for (ck.shards) |shard| {\n        const count = std.math.divCeil(usize, shard.map.size, page) catch unreachable;\n        const vec = try init.gpa.alloc(u8, count);\n        defer init.gpa.free(vec);\n        if (mincore(@ptrCast(shard.map.bytes.ptr), shard.map.size, vec.ptr) != 0) return error.Mincore;\n        for (vec) |v| resident += @intFromBool(v & 1 != 0);\n        pages += count;\n    }\n    if (resident != pages) return error.IncompleteResidency;\n    var hash = std.crypto.hash.sha2.Sha256.init(.{});\n    var bytes: u64 = 0;\n    for (ck.shards) |shard| {\n        hash.update(shard.map.bytes[0..shard.map.size]);\n        bytes += shard.map.size;\n    }\n    var digest: [32]u8 = undefined;\n    hash.final(&digest);\n    std.debug.print(\"load_s={d:.6} load_and_sha256_s={d:.6} bytes={d} sha256={s} resident={d} pages={d} touch_sum={d}\\n\", .{ loaded - start, mtl.clock.seconds() - start, bytes, std.fmt.bytesToHex(digest, .lower), resident, pages, sum });\n}\n","cache.c":"#include <fcntl.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/mman.h>\n#include <sys/stat.h>\n#include <unistd.h>\n\nint main(int argc, char **argv) {\n    if (argc < 3) return 2;\n    size_t total = 0, resident = 0;\n    for (int i = 2; i < argc; ++i) {\n        int fd = open(argv[i], O_RDONLY);\n        struct stat st;\n        if (fd < 0 || fstat(fd, &st)) return 3;\n        size_t size = (size_t)st.st_size;\n        unsigned char *map = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0);\n        if (map == MAP_FAILED) return 4;\n        if (!strcmp(argv[1], \"cold\") && msync(map, size, MS_INVALIDATE)) return 5;\n        if (!strcmp(argv[1], \"warm\")) {\n            unsigned char *buf = malloc(8 * 1024 * 1024);\n            if (!buf) return 6;\n            size_t offset = 0;\n            while (offset < size) {\n                size_t want = size - offset < 8 * 1024 * 1024 ? size - offset : 8 * 1024 * 1024;\n                ssize_t got = pread(fd, buf, want, (off_t)offset);\n                if (got <= 0) return 7;\n                offset += (size_t)got;\n            }\n            free(buf);\n        }\n        size_t page = (size_t)getpagesize();\n        size_t pages = (size + page - 1) / page;\n        char *vec = malloc(pages);\n        if (!vec || mincore(map, size, vec)) return 8;\n        size_t count = 0;\n        for (size_t j = 0; j < pages; ++j) count += (vec[j] & 1) != 0;\n        printf(\"%s resident=%zu pages=%zu\\n\", strrchr(argv[i], '/') + 1, count, pages);\n        total += pages;\n        resident += count;\n        free(vec);\n        munmap(map, size);\n        close(fd);\n    }\n    printf(\"total resident=%zu pages=%zu\\n\", resident, total);\n    if (!strcmp(argv[1], \"cold\")) return resident != 0;\n    return resident != total;\n}\n","build-approved.py":"import pathlib\nimport subprocess\nimport sys\nroot = pathlib.Path.cwd()\nout = root / '.zig-cache/h3-load-receipt'\nbase = pathlib.Path(sys.argv[1])\nfor label, tree in [('base', base), ('head', root)]:\n    dep, source = ('core', 'root.zig') if label == 'head' else ('safetensors', 'safetensors.zig')\n    cmd = ['nice', '-n', '10', 'zig', 'build-exe', '-O', 'ReleaseFast', '-lc', '-framework', 'Metal', '-framework', 'Foundation', '-framework', 'IOSurface', '-lobjc', '--dep', 'metal', '--dep', 'tensors', f'-Mroot={out}/load-approved.zig', '--dep', 'sys', '-O', 'ReleaseFast', f'-Mmetal={tree}/zig/src/metal/metal.zig', '--dep', 'metal', '--dep', dep, '--dep', 'sys', '-O', 'ReleaseFast', f'-Mtensors={tree}/zig/src/families/h3/tensors.zig', '--dep', 'sys', '-O', 'ReleaseFast', f'-M{dep}={tree}/zig/src/core/{source}', '-O', 'ReleaseFast', f'-Msys={tree}/zig/src/platform/sys.zig', f'-femit-bin={out}/{label}-approved']\n    subprocess.run(cmd, check=True)\nsubprocess.run(['clang', '-O2', '-Wall', '-Wextra', '-o', str(out/'cache'), str(out/'cache.c')], check=True)\n","run-approved.py":"import fcntl\nimport hashlib\nimport json\nimport os\nfrom pathlib import Path\nimport subprocess\nimport time\n\nroot = Path.cwd()\nout = root / '.zig-cache/h3-load-receipt'\nmodel = Path(os.environ['H3_TRANSFORMER'])\nshards = sorted(model.glob('model-*-of-00012.safetensors'))\nassert len(shards) == 12\nrecords = []\nlock = open('/tmp/tensorfold-model.lock', 'a')\nfcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)\nprint('shared model lock acquired', flush=True)\nfor rep in range(3):\n    order = ['base', 'head', 'off'] if rep % 2 == 0 else ['off', 'head', 'base']\n    for state in ['cold', 'warm']:\n        for label in order:\n            condition = subprocess.run([str(out / 'cache'), state, *map(str, shards)], text=True, capture_output=True, check=True)\n            verified = subprocess.run([str(out / 'cache'), 'cold' if state == 'cold' else 'verify', *map(str, shards)], text=True, capture_output=True, check=True)\n            executable = 'base-approved' if label == 'base' else 'head-approved'\n            mode = 'prefetch' if label == 'head' else 'sequential'\n            start = time.monotonic()\n            result = subprocess.run(['/usr/bin/time', '-l', str(out / executable), str(model), '12', mode], text=True, capture_output=True, check=True, env={**os.environ, 'TF_LOAD_PREFETCH': '0' if label == 'off' else '16'})\n            record = dict(rep=rep + 1, state=state, build=label, process_wall_s=time.monotonic() - start, conditioning=condition.stdout, verification=verified.stdout, stdout=result.stdout, stderr=result.stderr)\n            records.append(record)\n            (out / 'approved-runs.json').write_text(json.dumps(records, indent=2) + '\\n')\n            print(json.dumps({k: v for k, v in record.items() if k not in ['conditioning', 'verification']}), flush=True)\n            subprocess.run(['atc', 'pulse', '2999', '161332151983811', 'working', '--json'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n"},"runs":[{"rep":1,"state":"cold","build":"base","process_wall_s":611.5066934170001,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=587.996437 load_and_sha256_s=610.865484 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n      611.48 real        24.17 user        33.30 sys\n         66293039104  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1141  page reclaims\n             4045360  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  60  voluntary context switches\n             1337402  involuntary context switches\n        245552375271  instructions retired\n        243511328052  cycles elapsed\n            38714264  peak memory footprint\n"},{"rep":1,"state":"cold","build":"head","process_wall_s":55.823225417,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=17.661830 load_and_sha256_s=55.163306 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       55.81 real        21.81 user        17.39 sys\n         66428977152  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054820  page reclaims\n                  12  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                6376  voluntary context switches\n              462671  involuntary context switches\n        219339557106  instructions retired\n        167074589084  cycles elapsed\n           174717992  peak memory footprint\n"},{"rep":1,"state":"cold","build":"off","process_wall_s":664.5742743330001,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=613.601724 load_and_sha256_s=664.131583 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n      664.57 real        24.06 user        36.77 sys\n         66293940224  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1196  page reclaims\n             4045360  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  52  voluntary context switches\n             1731325  involuntary context switches\n        246940249609  instructions retired\n        259614759667  cycles elapsed\n            39599024  peak memory footprint\n"},{"rep":1,"state":"warm","build":"base","process_wall_s":37.62801800000011,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=8.092753 load_and_sha256_s=37.164556 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       37.59 real        24.46 user         4.11 sys\n         66293923840  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046538  page reclaims\n                  19  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               68479  involuntary context switches\n        162520832041  instructions retired\n        122332285188  cycles elapsed\n            39599000  peak memory footprint\n"},{"rep":1,"state":"warm","build":"head","process_wall_s":48.25697812499993,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=0.855063 load_and_sha256_s=47.823901 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       48.25 real        21.91 user        16.78 sys\n         66429059072  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054837  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               82572  involuntary context switches\n        150587705503  instructions retired\n        164949450433  cycles elapsed\n           174799912  peak memory footprint\n"},{"rep":1,"state":"warm","build":"off","process_wall_s":65.96702333300004,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=13.388794 load_and_sha256_s=65.513608 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       65.96 real        25.41 user         4.04 sys\n         66294120448  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046569  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               42396  involuntary context switches\n        162196938542  instructions retired\n        119148338524  cycles elapsed\n            39779248  peak memory footprint\n"},{"rep":2,"state":"cold","build":"off","process_wall_s":197.90108620799992,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=173.623205 load_and_sha256_s=197.519129 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n      197.89 real        25.70 user        22.06 sys\n         66293514240  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1172  page reclaims\n             4045360  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  53  voluntary context switches\n              957315  involuntary context switches\n        237675217358  instructions retired\n        195550065964  cycles elapsed\n            39189448  peak memory footprint\n"},{"rep":2,"state":"cold","build":"head","process_wall_s":51.29268537500002,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=17.542982 load_and_sha256_s=51.042177 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       51.28 real        21.73 user        18.26 sys\n         66428108800  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054768  page reclaims\n                  12  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                7164  voluntary context switches\n              342931  involuntary context switches\n        287163600955  instructions retired\n        170463396917  cycles elapsed\n           173849640  peak memory footprint\n"},{"rep":2,"state":"cold","build":"base","process_wall_s":190.518353875,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=167.617501 load_and_sha256_s=190.218300 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n      190.51 real        23.95 user        20.18 sys\n         66293874688  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1175  page reclaims\n             4045379  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  58  voluntary context switches\n             1002302  involuntary context switches\n        238691297836  instructions retired\n        188021257883  cycles elapsed\n            39566256  peak memory footprint\n"},{"rep":2,"state":"warm","build":"off","process_wall_s":35.96455274999994,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=11.957404 load_and_sha256_s=35.563073 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       35.95 real        24.39 user         4.88 sys\n         66293923840  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046538  page reclaims\n                  20  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  10  voluntary context switches\n               46572  involuntary context switches\n        177667000352  instructions retired\n        125163454984  cycles elapsed\n            39566232  peak memory footprint\n"},{"rep":2,"state":"warm","build":"head","process_wall_s":26.34493441700033,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=0.707479 load_and_sha256_s=26.048534 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       26.34 real        21.98 user        13.79 sys\n         66427977728  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054771  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               42799  involuntary context switches\n        150211435374  instructions retired\n        152757579634  cycles elapsed\n           173702160  peak memory footprint\n"},{"rep":2,"state":"warm","build":"base","process_wall_s":28.432804583000234,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=5.254956 load_and_sha256_s=28.143895 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       28.42 real        24.50 user         3.39 sys\n         66293399552  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046516  page reclaims\n                  13  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   2  voluntary context switches\n               52200  involuntary context switches\n        164139610568  instructions retired\n        118774956516  cycles elapsed\n            39123912  peak memory footprint\n"},{"rep":3,"state":"cold","build":"base","process_wall_s":76.44831841699988,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=54.462333 load_and_sha256_s=76.212030 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       76.44 real        23.72 user        17.29 sys\n         66293350400  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1160  page reclaims\n             4045360  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                  65  voluntary context switches\n              962496  involuntary context switches\n        236482262768  instructions retired\n        174552692783  cycles elapsed\n            39025560  peak memory footprint\n"},{"rep":3,"state":"cold","build":"head","process_wall_s":33.81923266700005,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=6.838389 load_and_sha256_s=33.573677 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       33.81 real        21.62 user        12.00 sys\n         66428665856  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054809  page reclaims\n                  32  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                7482  voluntary context switches\n              331291  involuntary context switches\n        200938352054  instructions retired\n        143645740298  cycles elapsed\n           174373880  peak memory footprint\n"},{"rep":3,"state":"cold","build":"off","process_wall_s":85.9153807079997,"conditioning":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=0 pages=332100\nmodel-00002-of-00012.safetensors resident=0 pages=356395\nmodel-00003-of-00012.safetensors resident=0 pages=334037\nmodel-00004-of-00012.safetensors resident=0 pages=334037\nmodel-00005-of-00012.safetensors resident=0 pages=356395\nmodel-00006-of-00012.safetensors resident=0 pages=334037\nmodel-00007-of-00012.safetensors resident=0 pages=334037\nmodel-00008-of-00012.safetensors resident=0 pages=356395\nmodel-00009-of-00012.safetensors resident=0 pages=334037\nmodel-00010-of-00012.safetensors resident=0 pages=334037\nmodel-00011-of-00012.safetensors resident=0 pages=356395\nmodel-00012-of-00012.safetensors resident=0 pages=283458\ntotal resident=0 pages=4045360\n","stdout":"","stderr":"load_s=63.736057 load_and_sha256_s=85.635321 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       85.91 real        23.70 user        22.89 sys\n         66293448704  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n                1166  page reclaims\n             4045360  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n              954831  involuntary context switches\n        236371094827  instructions retired\n        198379337225  cycles elapsed\n            39107504  peak memory footprint\n"},{"rep":3,"state":"warm","build":"base","process_wall_s":29.93458895799995,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=6.029278 load_and_sha256_s=29.629244 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       29.93 real        24.04 user         5.10 sys\n         66293825536  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046550  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               40076  involuntary context switches\n        188486746769  instructions retired\n        124627642010  cycles elapsed\n            39500696  peak memory footprint\n"},{"rep":3,"state":"warm","build":"head","process_wall_s":27.820842750000338,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=0.656038 load_and_sha256_s=27.533069 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=0\n       27.81 real        21.67 user        14.71 sys\n         66428551168  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4054805  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n               75033  involuntary context switches\n        175026732718  instructions retired\n        155970305557  cycles elapsed\n           174259192  peak memory footprint\n"},{"rep":3,"state":"warm","build":"off","process_wall_s":28.626764708999872,"conditioning":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","verification":"model-00001-of-00012.safetensors resident=332100 pages=332100\nmodel-00002-of-00012.safetensors resident=356395 pages=356395\nmodel-00003-of-00012.safetensors resident=334037 pages=334037\nmodel-00004-of-00012.safetensors resident=334037 pages=334037\nmodel-00005-of-00012.safetensors resident=356395 pages=356395\nmodel-00006-of-00012.safetensors resident=334037 pages=334037\nmodel-00007-of-00012.safetensors resident=334037 pages=334037\nmodel-00008-of-00012.safetensors resident=356395 pages=356395\nmodel-00009-of-00012.safetensors resident=334037 pages=334037\nmodel-00010-of-00012.safetensors resident=334037 pages=334037\nmodel-00011-of-00012.safetensors resident=356395 pages=356395\nmodel-00012-of-00012.safetensors resident=283458 pages=283458\ntotal resident=4045360 pages=4045360\n","stdout":"","stderr":"load_s=4.865987 load_and_sha256_s=28.348722 bytes=66279087736 sha256=f86338f7bd602839302958bdae604bac00899687d4b6834ebcdc141383ef7044 resident=4045360 pages=4045360 touch_sum=15539714538223899602\n       28.62 real        23.67 user         3.52 sys\n         66293366784  maximum resident set size\n                   0  average shared memory size\n                   0  average unshared data size\n                   0  average unshared stack size\n             4046521  page reclaims\n                   0  page faults\n                   0  swaps\n                   0  block input operations\n                   0  block output operations\n                   0  messages sent\n                   0  messages received\n                   0  signals received\n                   0  voluntary context switches\n              159474  involuntary context switches\n        163608419495  instructions retired\n        116555245196  cycles elapsed\n            39009176  peak memory footprint\n"}]}

[ashhart]

Thanks @plotarmordev. Items 2 and 4 are done well. read_ahead.zig now cuts each shard into checkpoint_read_ahead.chunk spans and starts core's reader with readers(), so TF_LOAD_PREFETCH=0 turns it off. It also checks the reader before Model.load continues (h3.zig:146, tensors.zig readAhead). test-h3-read-ahead runs at 16 readers and at 0, and it's in macos_adapters. The recipe section is written impersonally now.

Item 1 is still open. The new receipt times Checkpoint.open up to full residency, and its main arm is a single CPU loop that touches every mapped word. Main never runs that loop: Metal faults the pages in during the first forward. So 167.6 s against 17.5 s doesn't measure main's cold load. Please post the measurement from the first ask:

1. Main against this head, on the same Mac, alternating the two builds: the transformer tool's (tf-h3-dit) load time and total wall time at 2048 and 4096.
2. Cold, with the shards evicted before each run (your msync(MS_INVALIDATE) with a zero mincore count is fine), and warm. Warm matters most here. On main, a warm start reads nothing ahead, so the head's re-read of every shard (0.7 s in your table) is added time unless the first forward wins it back.
3. The output hashes (latent and PNG) from every run, main's and this head's.

Then please:

• retitle the PR to what it does, for example "h3: read the transformer shards ahead of their Metal mappings", with no timing in the title;

• replace the residency table and its notes in the "Cold start" section of docs/recipes/minimax-h3-metal.md with those numbers;

• squash the eight commits into one code commit and one docs commit. Some of the docs commits add fork timings that later commits remove again;

• update the description to the squashed head. It still names e22362c2, a merge of main, but the branch is now f880d70d3, rebased onto main.

• Synthena (Ash's Agent)

[plotarmordev]

Thanks, you're right about item 1: main never runs that touch loop, so the table didn't measure main's cold load. I've withdrawn it.

I can't run the measurement you asked for. Every H3 case I have is image-only, and tf-h3-dit refuses those on main and on this head alike: geometry.zig returns H3EmptyMedia without audio rows, and the image checkpoint has no audio_in.weight. I won't fabricate audio rows or cases, so I have no 2048/4096 load or wall times, cold or warm, and no latent or PNG hashes for either build. The warm cost you point out (one pread pass per shard that main doesn't do) is unmeasured too. The recipe's Cold start section now says that in place of the table. If you can share an audio-bearing case, I'll run main and this head alternately on the same Mac, cold and warm, and post the hashes.

The other items are done: the branch is squashed to one code commit (14c9c56f0) and one docs commit (4c09b49b4) on main 6c5cef89, with the code tree unchanged from f880d70d3. The title is "h3: read the transformer shards ahead of their Metal mappings", and the description matches head 4c09b49b4.

[ashhart]

Thanks @plotarmordev. The shared checked reader, TF_LOAD_PREFETCH=0, nightly test wiring, shorter recipe and two-commit history address those asks. With the residency table withdrawn, the description now states the measurement gap clearly.

The one remaining item is the actual tf-h3-dit receipt: alternating main/head load and total wall times at 2048 and 4096, cold and warm, with the latent/PNG hashes. Your H3EmptyMedia limitation is clear; this stays waiting for that receipt from an admitted audio-bearing case.

• Synthena (Ash's Agent)