Repository navigation
Conversation
Load the actual Qwen and Nemotron engines in separate timing and hashing processes. Read final weight buffers through an owned pinned destination, preserve it after failed completion, and distinguish a late device-byte control.
Reserve the full aligned Qwen read slot once at open and read MLX shard spans directly into it. Let Nemotron integrated read slots retain pinned bytes through DMA completion and quarantine. Keep discrete and temporary-vector paths unchanged. Recording-driver tests cover different file/mmap bytes, growing spans without reallocations, unaligned tails, completion and event failures, and partial allocation cleanup. Move host-test registration into the checkpoint helper. The recipe requires warm A/A and A/B/B/A with read-ahead off and a separate default-read-ahead pair. Device weights, load time and peak memory still need qualification. Refs #2043.
|
Closing this draft. #2282 (merged 7 Nov) already covers it for Qwen. It is the weight-upload part of #2043, and #2043 is closed. Since #2282, Qwen 3.5's CUDA loader reads each layer past the page cache into reused buffers ( The Nemotron half, where integrated read slots become owned pinned stages (
|
Part of #2043. On integrated GPUs (GB10), Qwen's weight upload reader now reserves its whole aligned pinned slot once, when it opens. Before, the slot could be allocated again as tensor pieces grew. Completion waits and the quarantine of failed copies stay as they were. Discrete-GPU paths and Nemotron's slot ownership are unchanged.
Three commits
A host regression grows a span piece by piece and requires one pinned allocation on this head.
docs/recipes/cuda-weight-upload.mddescribes the device run.Checks (host)
Not checked yet, and needed before it lands, on a GB10 for Qwen and Nemotron:
It stays a draft until that receipt is posted. If it shows no gain, it closes.
zig/build/cuda.zigends at 599 lines here. The GLM startup PR also adds one line there, so whichever lands second ends at 600.