Repository navigation
Conversation
8e8fc81 to
eebcf48
Compare
f88414b to
9905ddf
Compare
9905ddf to
14ea21e
Compare
14ea21e to
c6d1c72
Compare
Keep the direct-read switch and staging bound in core.direct_io so the pack reader applies to main alone. Preserve the mapped-byte comparisons, strided spans and shard-count coverage. Refs #1857.
…cache (core/direct_io)
Keep the relocated pack-range advice under the shared load-readiness policy and update its host source guard. Put the reviewed loading note at the top of Unreleased.
…ts O_DIRECT tests take the Linux path
efdeaa5 to
c2090ff
Compare
|
Rebased onto today's main (
|
|
Device check status: blocked for now. On main, the CUDA server refuses DeepSeek-V4.1 behind its whole-model gate (#827). Its prompt path still needs captured Triton kernels, so a served main-vs-branch run can't start. The EXL3 2.9 bpw checkpoint is 197 GB, so it needs two GB10s at TP=2; one box can't hold it. This PR waits for #827, or for a two-GB10 run once both boxes have room for the checkpoint. Host tests and CI stand as they are.
|
Part of #1857: DeepSeek-V4.1 reads its pack's weight spans past the page cache, so a load no longer fills the cache it then has to release.
What changes
Pack.readreads DeepSeek-V4.1's weight spans with O_DIRECT throughcore/direct_io, in at most 8 MiB of aligned staging, with exact strided gathers.core/direct_ionow owns theTF_LOAD_DIRECTswitch and the chunk size.TF_LOAD_DIRECT=0reads through the page cache as before. A file system that refuses O_DIRECT, at open or on a read, falls back the same way.pack_read.zig, so existing prepared folders rebuild once, the same on every rank.The pack hash and the prepared-image reads still go through the page cache, so this narrows #1857's minimum during load rather than fixing it outright.
Checks
TF_LOAD_DIRECTmodes. The lean check finds no new problems.core/direct_io, so it stands on main alone).c6d1c720):test-dsv41-direct-hostin bothTF_LOAD_DIRECTmodes,test-dsv41-env-hostandcore/direct_io's tests all pass, including the two that only run on Linux. The first run on the previous head failed one: under O_DIRECT, a pack cut short inside a span re-read the same aligned tail and reportedReadFailedinstead of end-of-file, so the pack reader never saidShortRead.c6d1c720fixes that and adds a test for it. A Mac never takes this path (no O_DIRECT).