A CLI and Python library for converting diffusion checkpoints and text encoders to native ComfyUI quantization formats. It selects eligible weights by architecture, preserves sensitive tensors, converts within a working-memory budget, and validates reconstruction before publishing the output.
Optional MSE optimization searches scales and clipping on each format's actual numerical grid. Inference uses the existing ComfyUI/Kitchen layouts and kernels. The target model architecture must be supported by your ComfyUI installation; some architectures require their own native extensions.
Use Python 3.12–3.13 with your intended PyTorch build already installed. From this checkout, install the package and native-layout dependency:
python -m pip install '.[kitchen]'
comfyquant --version
comfyquant --no-config --list-formatsYou can also install a release wheel with python -m pip install /path/to/package.whl.
PyTorch is supplied by your environment. Add the hub extra for Hugging Face
inputs, or dev for development and packaging tools. See the
installation guide for command paths, completions and defaults.
Inspect the checkpoint and preview the selected layers:
comfyquant --no-config input.safetensors --inspect
comfyquant --no-config input.safetensors --format int8-convrot --dry-runConvert, validate against the source, then verify the serialized output:
comfyquant --no-config input.safetensors --format int8-convrot \
--output output-int8.safetensors --max-memory 512M --validate
comfyquant --no-config --verify-output output-int8.safetensorsThe source is preserved. Replacing an output requires --overwrite. Unsupported
shapes and protected tensors keep their source precision. Review the dry-run and
report for actual coverage; a format name does not guarantee native acceleration
on every consumer, shape or backend.
Enable --optimizer mse for eligible quantized weights in any registered
architecture. The default remains --optimizer none. The original encoding is
always considered, and the optimizer retains it when no valid candidate improves
the selected objective.
For Z-Image INT4 ConvRot with weight reconstruction MSE:
comfyquant --no-config z-image.safetensors --architecture z_image \
--format int4-convrot --optimizer mse --mse-objective weight \
--device auto --max-memory 512M --output z-image-int4-mse.safetensors --validateFor mixed precision with recorded activations:
comfyquant --no-config input.safetensors --format mixed --profile balanced \
--optimizer mse --mse-objective activation \
--calibration-source activations.npz --require-calibration \
--max-memory 512M --output output-mixed.safetensors --validate| Objective | Selection |
|---|---|
weight |
Minimize weight reconstruction error |
activation |
Minimize reconstructed-weight output error on recorded activation rows; complete coverage required |
auto (default) |
Use activation error where samples exist and weight error elsewhere |
--mse-steps controls applicable scale/clipping searches; --mse-iterations
controls applicable least-squares refinement. Searches respect integer rounding,
FP8 scale storage and block-format representations. Improvement depends on the
weights, format and objective; reconstruction MSE does not establish image quality.
See MSE optimization for controls, calibration,
benchmarks and numerical limits.
For measured W4A8/W6A8 precision exchanges, held-out calibration, optional W6/INT8 quality references and whole-file compression limits, see adaptive mixed precision. The adaptive profile is opt-in; its calibration gates do not establish near-lossless model quality.
| Format | Stored weights and execution request |
|---|---|
int8, int8-convrot |
INT8 with FP32 row scales; optional ConvRot |
w4a4 |
Packed INT4 with FP32 row scales; INT8 execution request by default |
int4-convrot |
Packed INT4 with ConvRot and an INT4 execution request |
w4a8 |
Codebook INT4 with FP8 group scales and FP32 channel scales; INT8 decode grid |
w6a8 |
Uniform six-bit weights with FP8 group scales and FP32 channel scales; INT8 decode grid |
mixed |
Per-layer W4A4/W4A8/INT8; optional W6A8 and INT8 ConvRot candidates |
fp8-e4m3, fp8-e5m2 |
FP8 with one FP32 tensor scale |
nvfp4 |
E2M1 values, E4M3 block scales and an FP32 tensor scale |
mxfp8 |
E4M3 values and E8M0 block scales |
fp16, bf16 |
Precision casts; no MSE quantization-scale optimization |
W6A8 requires Kitchen's explicit six-bit API and native ComfyUI registration. Block formats require compatible layout APIs; accelerated inference needs an appropriate backend. See support and W6A8.
List policies with comfyquant --no-config --list-architectures. Detection uses
tensor names and shapes, with separate policies for distinct model variants.
--architecture NAME provides an explicit override. Include/exclude controls
still obey shape, precision and consumer safeguards.
Inputs can be safetensors files, indexed shards or model directories. Pickle
input requires --trust-pickle. Hugging Face input requires the hub extra and
explicit download options; --offline prevents model downloads.
Use --components text_encoder for supported text towers. Token embeddings stay
at source precision unless explicitly enabled with --quantize-text-embeddings
for supported INT8/mixed paths, or selected by an exact reference recipe. Norms, modulation, input/output boundaries and
protected auxiliary towers retain their architecture policies.
Native Flux 3 Klein and its step-distilled flow-map variant use the separate
experimental flux3 policy, including fused or split QKV block linears. See the
Flux 3 Klein guide for preserved tensors and evidence
limits. Flux 2 Klein remains under flux2.
Native Fibo2 and MOSS-Image policies follow the official repacked layouts.
Use --reference-recipe to reproduce a pinned
Comfy-Org artifact's exact layer selection and per-layer 16/64/256 ConvRot settings.
The metadata-only extractor checks layouts without downloading model weights.
--max-memory budgets temporary conversion work, excluding mmap pages, framework
overhead and allocator caches. W4A8, W6A8 and mixed conversions support matching
resume journals. Other formats restart atomically. NVFP4/MXFP8 require a sufficient
whole-layer budget. These limits describe conversion, not inference VRAM.
from native_quantizer import api
from native_quantizer.options import ConversionOptions
status = api.convert(ConversionOptions(
model="input.safetensors", output="output.safetensors",
format="int8-convrot", optimizer="mse", mse_objective="weight",
max_memory="512M", validate=True,
))
verification = api.verify_output("output.safetensors")API requests are explicit and do not read CLI preferences. See architecture and API for operations and module ownership.
native_quantizer/ CLI/library implementation and packaged self-test fixtures
tests/ Regression suites (cpu/, cuda/) and shared support
scripts/ Checks, installed smoke, benchmarks and Linux setup
docs/ Usage, numerical contracts, support and release guides
docs/models/ Architecture-specific conversion guides
.github/ CI and contribution templates
The root Python entry points remain thin compatibility interfaces. Maintained
implementation lives in native_quantizer/; dependencies are declared in
pyproject.toml. Generate a custom architecture starter with
comfyquant --no-config --write-architecture-template policy.json.
- Installation and configuration
- MSE objectives and benchmarks
- Mixed precision, calibration and runtime costs
- Flux 3 Klein, Iris, Kandinsky 6, Ming-Image, Qwen-Image 2.1, MiniCPM-V 4.7 1B, LFM3-VL-2B
- Support and validation
- Contributing, release process and changelog
python scripts/check.py
python scripts/check.py --comfy-root /path/to/ComfyUI --cudaDefault checks use CPU fixtures with isolated preferences. Requested consumer and CUDA checks fail when prerequisites are missing. Synthetic tests establish serialization, numerical and consumer contracts; model quality, workflow latency, offload and adapters need separate evaluation. Development checks do not download model weights or start services.
Licensed under Apache-2.0.