Repository navigation
core/direct_io: an O_DIRECT read past a file's end is EndOfFile, not ReadFailed; Nemotron's CUDA checkpoint reader tested on the host - #2602
Merged
Conversation
This was referenced Nov 8, 2026
…e end of the file, not a failed read On Linux, an O_DIRECT read of a file that ends inside the aligned span retries from the last 4 KiB boundary, reads the same tail again and made no progress, which readAt reported as ReadFailed. It is end of file: callers that name a short file (DeepSeek's pack reader says ShortRead) or stop on EndOfFile (Nemotron's CUDA checkpoint reads) now get it. macOS has no O_DIRECT, so a Mac never took this path; it showed on Linux ext4.
families/nemotron/cuda_source.zig reads checkpoints with io_uring and O_DIRECT on Linux and on threads elsewhere, and no test reached it. cuda_source_test.zig drives Source against a fake CUDA driver that logs each host-to-device copy and stream wait: two shards' tensors at unaligned offsets and sizes, one split over two slot reads, more uploads than slots, through the io_uring ring and the thread fallback. Every device byte must be the file's and every copy waited for. Each test asserts which path ran, and skips with a printed reason where io_uring or O_DIRECT is refused. The third test shortens a shard after open, so a tensor's bytes run past its end. It wants EndOfFile with nothing copied, on every path and read mode. Before the direct_io change in the previous commit, O_DIRECT's aligned rule gave ReadFailed: the retry from the last 4 KiB boundary re-read the same tail. Nothing was copied and nothing hung either way.
ashhart
force-pushed
the
lead/direct-io-eof
branch
from
November 8, 2026 20:43
ab36007 to
cc79b02
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Linux,
core/direct_io's O_DIRECT read of a file that ends inside the aligned span it asked for retries from the last 4 KiB boundary, reads the same tail again, makes no progress, and reportsReadFailed. It is the end of the file, and callers handle that: Nemotron's CUDA checkpoint reads stop onEndOfFile, and DeepSeek's pack reader (in #1924) names itShortRead. Nothing was copied wrongly and nothing hung before; the error was wrong. macOS has no O_DIRECT, so a Mac never took this path.What changes
core/direct_io.readAt: an aligned retry that makes no progress returnserror.EndOfFile. Only an aligned retry can make no progress (an unaligned read continues fromgot), so no other case changes. A unit test pins it with a scriptedpread.families/nemotron/cuda_source_test.zig(new): Nemotron's CUDA checkpoint reader (cuda_source.zig), which no test reached, against a fake CUDA driver that logs each host-to-device copy and stream wait. Two shards' tensors at unaligned offsets and sizes, one split over two slot reads, more uploads than slots, through both the io_uring ring and the thread fallback; every device byte must be the file's. Each test asserts which path ran and skips with a printed reason where io_uring or O_DIRECT is refused. A third test shortens a shard after open so a tensor runs past its end, and wantsEndOfFilewith nothing copied, on every path and read mode. It runs intest-cuda-host(andzig build teston Linux and Windows).Checks (Linux aarch64, ext4)
core/direct_io's tests pass (5/5, including the two that only run on Linux).test-cuda-hostotherwise: the only failures are twodrama_voice_tooldirectory-permission tests, which fail identically on main there because that machine runs tests as root.test-cuda-host335/335 steps, 0 failed. The thread-path test passes through unbuffered reads (FILE_FLAG_NO_BUFFERING, the aligned path), the io_uring test skips (no io_uring), and the shortened-shard test skips (Windows can't shorten a mapped file), the same on main plus the test. The fullzig build testshows no new failures.zig fmt --checkclean.