[ElderOrb]

A synchronous hipMemcpyHtoD pins the host source for the lifetime of the blit. A single multi-gigabyte source fails to pin, so a large checkpoint cannot be uploaded at all. Copy in 64 MiB windows instead: the pin stays bounded, and a copy no larger than a window still takes one call.

Split out of #2490 at review request: this is the one change there that is not ModelOpt-specific.

Ran: HIP on Windows, gfx1151, a 27B ModelOpt checkpoint (a single 20+ GiB upload) loads and serves.
Not run: any Linux HIP target, any other gfx target.

[ashhart]

@ElderOrb thanks for splitting this out. The windows stay inside the range span already checks, the last partial window is exact, and a small upload is still one call, so it's the same bytes as before. One thing: please add a CHANGELOG line under Unreleased, e.g. "HIP: a large host upload copies in 64 MiB windows, so a multi-gigabyte tensor no longer fails to pin (#2584). Thanks to @ElderOrb."

[ashhart]

@ElderOrb, a small housekeeping change so this can land: the commit carried a personal email address as its author, and we keep those out of the public history. I re-authored it to your GitHub noreply address (23638+ElderOrb@users.noreply.github.com, the one your earlier commits here use), with your name and the same tree, and pushed it to your branch as ee891d25ad. Nothing else changed. If you set that address in git (git config user.email) for this repository, future commits won't need it.

[ashhart]

Thank you @ElderOrb. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @ElderOrb! This is in TensorFold 1.2.6, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.6

• Synthena (Ash's Agent)