Repository navigation
Conversation
pdf_worker_test.zig was wired only in the macOS Metal graph, so the worker's Linux paths (the /proc/self/statm address-space cap and the close_range sweep in the spawned tensorfold-native) ran nowhere. The CUDA side's server host steps now define test-pdf-worker where the Metal build does not, spawning the worker from this host's CUDA tensorfold-native, as metal_targets.zig does from the Metal one. cache_fit.zig was built only into Metal targets, and its no-footprint test skips on macOS, so it ran on no platform. test-cache-fit-host builds it over the host metal module that test-metal-residency-host already runs on Linux. Both steps join the nightly linux_host shard; the per-PR test step is unchanged.
…e end of the file, not a failed read On Linux, an O_DIRECT read of a file that ends inside the aligned span retries from the last 4 KiB boundary, reads the same tail again and made no progress, which readAt reported as ReadFailed. It is end of file: callers that name a short file (DeepSeek's pack reader says ShortRead) or stop on EndOfFile (Nemotron's CUDA checkpoint reads) now get it. macOS has no O_DIRECT, so a Mac never took this path; it showed on Linux ext4.
families/nemotron/cuda_source.zig reads checkpoints with io_uring and O_DIRECT on Linux and on threads elsewhere, and no test reached it. cuda_source_test.zig drives Source against a fake CUDA driver that logs each host-to-device copy and stream wait: two shards' tensors at unaligned offsets and sizes, one split over two slot reads, more uploads than slots, through the io_uring ring and the thread fallback. Every device byte must be the file's and every copy waited for. Each test asserts which path ran, and skips with a printed reason where io_uring or O_DIRECT is refused. The third test shortens a shard after open, so a tensor's bytes run past its end. It wants EndOfFile with nothing copied, on every path and read mode. Before the direct_io change in the previous commit, O_DIRECT's aligned rule gave ReadFailed: the retry from the last 4 KiB boundary re-read the same tail. Nothing was copied and nothing hung either way.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI for a landing train: #2599 #2602 stacked on main in that order. Nothing merges from this PR: each PR above lands on its own once this tip is green, and this PR is then closed.