Skip to content

core/direct_io: an O_DIRECT read past a file's end is EndOfFile, not ReadFailed; Nemotron's CUDA checkpoint reader tested on the host - #2602

Merged
ashhart merged 3 commits into
mainfrom
lead/direct-io-eof
Nov 8, 2026
Merged

ashhart merged 3 commits into
mainfrom
lead/direct-io-eof

Conversation

@ashhart

@ashhart ashhart commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

On Linux, core/direct_io's O_DIRECT read of a file that ends inside the aligned span it asked for retries from the last 4 KiB boundary, reads the same tail again, makes no progress, and reports ReadFailed. It is the end of the file, and callers handle that: Nemotron's CUDA checkpoint reads stop on EndOfFile, and DeepSeek's pack reader (in #1924) names it ShortRead. Nothing was copied wrongly and nothing hung before; the error was wrong. macOS has no O_DIRECT, so a Mac never took this path.

What changes

  • core/direct_io.readAt: an aligned retry that makes no progress returns error.EndOfFile. Only an aligned retry can make no progress (an unaligned read continues from got), so no other case changes. A unit test pins it with a scripted pread.
  • families/nemotron/cuda_source_test.zig (new): Nemotron's CUDA checkpoint reader (cuda_source.zig), which no test reached, against a fake CUDA driver that logs each host-to-device copy and stream wait. Two shards' tensors at unaligned offsets and sizes, one split over two slot reads, more uploads than slots, through both the io_uring ring and the thread fallback; every device byte must be the file's. Each test asserts which path ran and skips with a printed reason where io_uring or O_DIRECT is refused. A third test shortens a shard after open so a tensor runs past its end, and wants EndOfFile with nothing copied, on every path and read mode. It runs in test-cuda-host (and zig build test on Linux and Windows).

Checks (Linux aarch64, ext4)

  • Main plus only the new test: the shortened-shard test fails on both paths, "io_uring reads through O_DIRECT gave ReadFailed past the shard's end, want EndOfFile" and the same for threads.
  • This branch: it passes, and core/direct_io's tests pass (5/5, including the two that only run on Linux).
  • test-cuda-host otherwise: the only failures are two drama_voice_tool directory-permission tests, which fail identically on main there because that machine runs tests as root.
  • Native Windows (x86_64): test-cuda-host 335/335 steps, 0 failed. The thread-path test passes through unbuffered reads (FILE_FLAG_NO_BUFFERING, the aligned path), the io_uring test skips (no io_uring), and the shortened-shard test skips (Windows can't shorten a mapped file), the same on main plus the test. The full zig build test shows no new failures.
  • On a Mac the Linux paths skip, as expected; zig fmt --check clean.

…e end of the file, not a failed read

On Linux, an O_DIRECT read of a file that ends inside the aligned span retries from the last 4 KiB boundary, reads the
same tail again and made no progress, which readAt reported as ReadFailed. It is end of file: callers that name a short
file (DeepSeek's pack reader says ShortRead) or stop on EndOfFile (Nemotron's CUDA checkpoint reads) now get it. macOS
has no O_DIRECT, so a Mac never took this path; it showed on Linux ext4.
families/nemotron/cuda_source.zig reads checkpoints with io_uring and
O_DIRECT on Linux and on threads elsewhere, and no test reached it.
cuda_source_test.zig drives Source against a fake CUDA driver that
logs each host-to-device copy and stream wait: two shards' tensors at
unaligned offsets and sizes, one split over two slot reads, more
uploads than slots, through the io_uring ring and the thread fallback.
Every device byte must be the file's and every copy waited for. Each
test asserts which path ran, and skips with a printed reason where
io_uring or O_DIRECT is refused.

The third test shortens a shard after open, so a tensor's bytes run
past its end. It wants EndOfFile with nothing copied, on every path and
read mode. Before the direct_io change in the previous commit, O_DIRECT's
aligned rule gave ReadFailed: the retry from the last 4 KiB boundary
re-read the same tail. Nothing was copied and nothing hung either way.
@ashhart
ashhart force-pushed the lead/direct-io-eof branch from ab36007 to cc79b02 Compare November 8, 2026 20:43
@ashhart
ashhart merged commit f333dd0 into main Nov 8, 2026
2 of 10 checks passed
@ashhart
ashhart deleted the lead/direct-io-eof branch November 10, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant