Repository navigation
Support for AMD GPUs (Vulkan or ROCm/HIP) for Edge0 120B / 145B #197
Description
Area
New backend (GPU / NPU / other accelerator)
What problem are you trying to solve?
Hi, thanks for releasing Edge0 as open weights!
The v0.2.0 release notes list Edge0 145B / 120B as NVIDIA-only (CUDA 13, compute capability 7.5+). Edge0 35B already runs on Windows via Vulkan.
Are there plans to support the 120B / 145B models on AMD GPUs, either through the existing Vulkan backend or via ROCm/HIP?
My setup:
- GPU: AMD Radeon RX 9070 XT (RDNA4, gfx1201, 16 GB VRAM)
- CPU: Ryzen 5 9600X, 32 GB RAM
- Windows 11
This would meet the stated 16 GiB GPU / 20 GiB system memory minimums apart from the vendor requirement. Is the edge0-infer source available, so the community could help with a port? Happy to test a preview build.
Thanks!
Proposed solution
No response
Alternatives considered
No response
Reacted by tom, Gangadhar and Rene Leonhardt
Reacted by tom and Gangadhar
Activity
shuk777 commentedon Nov 13, 2026
Thanks for your interest in our work! Our next goal is of course supporting more environments, thanks for offering to help test. There is already a pr #196 focusing on AMD support, our team will review it immediately. Will let you know once AMD support is ready!
gangadharjannu commentedon Nov 13, 2026
Mine is R9700 AI PRO card, I am also happy to test
JuggyBob commentedon Nov 13, 2026
Adding a second AMD data point, since PR #196 (#196) covers gfx1100 with 24 GB VRAM and this is a 16 GB card.
Machine (runs upstream Strata's HIP build today):
For context on what gfx1102 already does here: this box serves Qwen3.8-Flash-Next Coder IQ1_M (23.4 GiB of expert weights) on upstream Strata's
hip-rdna3engine build at 39.5–44.4 tok/s decode and 320–341 tok/s prefill at 256K context, so the RDNA3 wave32 path is working, not just compiling.Questions for the AMD branch:
STRATA_HIP_ARCHS=gfx1102build cleanly, or is anything in PR feat(strata): build and run edge0-145b on AMD HIP (Windows, gfx1100) #196 gfx1100-specific? Same generation, but the CU count and L2 differ.--mmap-experts/--resident-budget-gib, or is 145B effectively out of scope below ~24 GB VRAM? Edge0-120B-q3k (27.98 GiB) fits this machine's RAM comfortably by comparison, so it may be the realistic target here.STRATA_LATENT_FUSED=0,STRATA_LAT_FUSED_Q3K=0as in the PR)?What I can contribute if useful: a gfx1102 build,
tools/strata_bench.pynumbers for 145B-q3k and 120B-q3k at 16 GB VRAM / 31 GB RAM, and a sweep of the residency settings at that budget. I have no NVIDIA card, so I cannot provide a CUDA comparison.I have not built or run anything yet — the numbers above come from upstream Strata, not from Edge0.