Add LTXAV model detection monkey-patch for checkpoint compatibility#482
Open
Marcel (goodguy1963) wants to merge 1 commit into
Open
Add LTXAV model detection monkey-patch for checkpoint compatibility#482Marcel (goodguy1963) wants to merge 1 commit into
Marcel (goodguy1963) wants to merge 1 commit into
Conversation
Monkey-patches comfy.model_detection.detect_unet_config to auto-detect missing LTXAV parameters from the checkpoint: - cross_attention_adaln (from prompt_scale_shift_table existence) - audio_cross_attention_dim & audio_attention_head_dim (from audio_attn2 weights) - Embeddings connector dimensions (from learnable_registers shapes) Fixes 'size mismatch' RuntimeError when loading LTXAV GGUF checkpoints that were trained with cross_attention_adaln=True and different connector head dimensions than the core defaults.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Loading LTXAV GGUF checkpoints fails with
RuntimeError: size mismatchbecause the ComfyUI core model detection doesn't auto-detect these parameters from the checkpoint:cross_attention_adaln— checkpoint uses 9 ada params per block, core defaults to 6audio_attn2weightsFix
model_detection_patch.py— a zero-touch monkey-patch applied at ComfyUI-LTXVideo import time that wrapscomfy.model_detection.detect_unet_config. When an LTXAV model is detected, it reads the missing parameters from the state dict keys:cross_attention_adalnprompt_scale_shift_tablekey existenceaudio_cross_attention_dimaudio_attn2.to_k.weightshape[1]audio_attention_head_dimaudio_attn2.to_k.weightshape[0] ÷ 32learnable_registersshape[1] ÷ 32No core ComfyUI files are modified — the patch lives entirely within the custom node and activates via a one-line import in
__init__.py.Testing
Verified model creation produces exact checkpoint-matching shapes: