cuda-graphs

Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.

From-scratch, heavily-annotated CUDA inference runtime for Qwen2.5-Coder-7B on H100 (sm_90). Custom INT4 packer, fused GEMV, paged KV, split-KV attention, CUDA graph decode — every hot path commented for the why. Educational, not a llama.cpp replacement.

GLM-5.2 on 4x DGX Spark with adaptive MTP K2/K4/K5, FULL CUDA graphs, DCP2, 520K context, and a downloadable ARM64 runtime image.

纯 CUDA 实现 GPT2 推理，逐步优化算子与架构，最终在单请求下推理速度领先 vLLM 约 1.5 倍。含详细 NCU 分析与 README 优化报告。

Deterministic sparse-index localization for Apple silicon (MLX/Metal) and CUDA/Triton, with stable ordering, a Python oracle, and reproducible evidence.

Measured serving recipes for DeepSeek-V4.1-Flash on 4x NVIDIA DGX Spark (GB10): 1M context on vLLM (CUDA graphs, vision, tools, DSpark) and a switchless-ring SGLang TP4 lane, plus a cross-project reference table. EN + 中文.

A drop-in, hardware-agnostic library for Fused Triangle Multiplicative Updates across AlphaFold3-family models, powered by CUTLASS CuTe kernels.

Hardware-aware Ollama fork with legacy NVIDIA support, controllable runtime tuning, Q4/Q8 KV cache, Flash Attention, CUDA Graphs and optimized multi-GPU inference.

A production-ready, high-performance CUDA Runtime and Driver API library for Zig.

fable-5's improvements to the CIFAR-10 speedrun over the hiverge baseline, measured

QSA HiSparse as an SGLang plugin: 256K CPU KV offload and text/image host prefix reuse for Qwen Sparse Attention on unmodified SGLang

Abliterated DeepSeek-V4.1-Flash-1017, 3.84 bpw EXL3, on 3x NVIDIA DGX Spark with tensor parallelism (TP3). TensorFold engine, deploy scripts, OpenAI-compatible API, benchmarks. 3,816 tok/s prefill.

Optimized CSM-1B TTS pipeline for RTX 5090 (Blackwell sm_120). CUDA graph replay via patched HF Transformers. ~0.46x RTF. Topics (tags): csm text-to-speech rtx-5090 blackwell cuda-graphs torch-compile sesame streaming pytorch

CUDA-graph-safe per-request activation steering plugin for vLLM. pip install hotwire-vllm

Ultra-High-Throughput Transformers Engine: NVIDIA FP8/TMA, Zero-Allocation Slotted Static KV Cache, Sub-5µs CUDA Graphs, Fused Logits Sampling, Tree Speculative Decoding, Split-KV Flash-Decoding & Apple Silicon Metal (MSL) Simdgroup Kernels

Bake neural rendering instead of running it every frame: a GPU-resident cache replaces per-frame neural inference, 20x to 34x faster at up to 48.7 dB PSNR. Rust, cuTile, RTX 4070 Ti.

Stencil-aware multi-GPU Conjugate Gradient solver (CUDA + MPI, NCCL, NVSHMEM), measured on 8× A100-SXM4-80GB. SpMV 2.08× faster than cuSPARSE CSR, CG 1.44× faster than NVIDIA AmgX at equal algorithm, 93.5% strong scaling efficiency (2D). Profiled with Nsight Systems and Nsight Compute.

From-scratch LLM inference engine for a free NVIDIA T4: paged KV cache, continuous batching, radix prefix cache, custom Triton paged-attention kernels, CUDA graphs. 4.98x HF generate() throughput at 32 concurrent requests, token-identical in fp32.

Real-time multi-camera object detection with every stage on the GPU: NVDEC decode, a fused CUDA preprocessing kernel, TensorRT FP16/INT8 from CUDA graphs, batched GPU NMS. 2,264 fps across 32 cameras on one RTX 5070 Ti, with each stage's latency measured.