HunyuanVideo: FP8 vs BF16, ComfyUI Setup, LoRA Training

TL;DR

• Use the FP8 UNET (pruned) if you have ≤24 GB VRAM; the BF16 UNET forces block-swapping even on a 4090.
• In ComfyUI native, load hunyuan_video_t2v_720p_bf16.safetensors as the diffusion model, clip_l.safetensors + llava_llama3_fp8_scaled.safetensors as text encoders, and hunyuan_video_vae_bf16.safetensors as the VAE.
• Do not enable --highvram — it pins CLIP to GPU and defeats CPU offload.
• LoRA training works on Linux via diffusion-pipe (config.toml); Windows DeepSpeed is broken — train in WSL2 or a VM.

Which HunyuanVideo model file should I download for my VRAM?

FP8 UNET for ≤24 GB VRAM; BF16 UNET for high-VRAM cards. The official Tencent model card lists 60 GB minimum VRAM for 720p generation with the base model. The BF16 variant exceeds that, so ComfyUI will block-swap to system RAM — the Civitai guide explicitly states "Even 4090 users will end up block swapping to the CPU if the BF16 model is used."

^[1] https://huggingface.co/tencent/HunyuanVideo ^[2] https://civitai.com/articles/13522

Full model file catalog with Civitai IDs

Source: https://civitai.com/articles/13522

Why does BF16 UNET block-swap on a 4090?

The BF16 UNET weights are the full-size model. With the VAE, two text encoders, and activations, peak GPU memory exceeds the VRAM available. ComfyUI then offloads transformer blocks to CPU (block-swapping), which adds latency per step. The FP8 UNET is the quantized version, so the entire model stack fits in 24 GB with room for activations — no offload needed.

Mechanism: The official HunyuanVideo repo lists 60 GB minimum VRAM for the base model at 720p. A 4090 has 24 GB VRAM, so the full BF16 model cannot fit. ComfyUI's block-swapping moves transformer layers between CPU and GPU per forward pass, adding ~2-3× step latency. The FP8 pruned UNET reduces weight memory by ~50%, bringing the total stack under 24 GB.

What ComfyUI model files go where?

The ComfyUI native workflow (announced December 20, 2024) expects four files in specific folders:

Source: https://blog.comfy.org/p/hunyuanvideo-native-support-in-comfyui

Should I use the --highvram flag?

No. The --highvram flag tells the launcher to keep all models in GPU memory. For users with ≤24 GB VRAM this forces the CLIP and text encoder onto the GPU, leaving no room for the UNET activations. The Civitai guide is explicit: "DO NOT USE HIGHVRAM FLAG if this is set you will force the CLIP to be in GPU. Normal Users DO NOT USE THIS FLAG."

Mechanism: --highvram disables the default CPU offload behavior. CLIP is small (~1 GB) but when pinned to GPU it reduces the activation budget for the 13 B-parameter UNET. Without --highvram, ComfyUI offloads CLIP and the text encoder to CPU automatically, keeping only the active UNET blocks on GPU.

What do --fp32-text-enc and --fp32-vae do?

They override the default FP16 load dtype for the text encoder and VAE respectively. Without them, ComfyUI downcasts FP32 weights to FP16 at load time. The flags are passed at launch:

python main.py --fp32-text-enc --fp32-vae --use-sage-attention

Mechanism: On CPU, FP16 arithmetic is emulated (upcast to FP32), so FP32 weights incur no speed penalty and can be faster than emulated FP16. The flags set the dtype at model load time; the offload device (CPU) then computes at that precision. Load time increases slightly due to larger weight tensors.

How do I train a HunyuanVideo LoRA?

Training uses diffusion-pipe with a config.toml pointing to the model files:

[model]
type = 'hunyuan-video'
transformer_path = '/path/to/hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors'
vae_path = '/path/to/hunyuan_video_vae_bf16.safetensors'
llm_path = '/path/to/llm'
clip_path = '/path/to/clip'

One user reported ~20 GB VRAM at 768 px resolution on Linux. Windows DeepSpeed is not functional — "cant download deepspeed on windows 10" and "windows deepspeed seems an impassable sinkhole" per community comments. Train in WSL2 or a Linux VM.

Worked example: training config.toml for 768 px on Linux

[model]
type = 'hunyuan-video'
transformer_path = '/mnt/data/models/hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors'
vae_path = '/mnt/data/models/hunyuan_video_vae_bf16.safetensors'
llm_path = '/mnt/data/models/llava_llama3_fp8_scaled'
clip_path = '/mnt/data/models/clip_l'

[train]
resolution = 768
batch_size = 1
learning_rate = 1e-4
max_steps = 2000
mixed_precision = 'fp16'
gradient_accumulation_steps = 4

Notes from community reports:

• Paths use /mnt/c/ prefix on WSL2 to reference Windows C: drive (e.g., /mnt/c/Users/name/models/...)
• Resolution 768 px consumed ~20 GB VRAM on a Linux box (single data point)
• The filename hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors appears in the training config example but was not independently verified on HuggingFace

Source: https://civitai.com/articles/9798/training-a-lora-for-hunyuan-video-on-windows (comments by ymz0023432, jhnprst2150)

How do I use a trained LoRA in ComfyUI?

Add a Load LoRA node between Load Diffusion Model and the Sampler / CLIP / Schedule nodes. The LoRA weights are applied to the diffusion model before sampling. One user confirmed: "just add the Load Lora node between the LoadDiffusionModel and the Sampler/Clip/Schedule nodes. At least seems to work and delivers a different video."

LoRA insertion point in native workflow

Load Diffusion Model → Load LoRA → Sampler / CLIP / Schedule → VAE Decode

Source: https://blog.comfy.org/p/hunyuanvideo-native-support-in-comfyui (community comment by jhnprst2150)

Common errors and fixes

Practical caveats from community reports

1. The --bf16-unet flag in example scripts is not a documented ComfyUI flag. The Civitai article's example .bat includes --bf16-unet --use-sage-attention, but neither flag appears in official ComfyUI or HunyuanVideo documentation. Model precision is controlled by which safetensors file you load, not a CLI flag.

1. Scaled FP8 text encoder requires 24 GB VRAM + --highvram. The source states: "Using a Scaled FP8 TE is not recommended unless you have 24GB of VRAM and are using the HighVram Flag." The standard TE (llava_llama3_fp8_scaled.safetensors) is the FP8 variant used in the native ComfyUI workflow.

1. FP8 VAE is not recommended. The source explicitly says "I do not recommend the FP8 VAE" without explaining why. The BF16 pruned VAE (Civitai 1489354) is the recommended choice for most users.

1. Windows LoRA training is not viable. DeepSpeed lacks Windows support. The Civitai article 9798 was titled "Training a LoRA for Hunyuan video on Windows" but was marked "Reinstating soon" — commenters confirmed actual training was done on Linux.

1. The 8 GB workflow JSON exists but is impractical. Attached to Civitai article 13522. A ComfyUI blog commenter warns: "Don't even bother to try this with less than 24 or 40 GB VRAM." Expect very slow generation and frequent OOM.

1. diffusion-pipe tool identity uncertain. The training config references diffusion-pipe with type = 'hunyuan-video'. The kohya-ss/diffusion-pipe repository returns 404 on GitHub; Kohya's primary training repo is kohya-ss/sd-scripts. Whether this is a separate project, fork, or misnamed reference is unverified.

1. Training VRAM requirements are not well-established. One user reported ~20 GB at 768 px (single data point). No systematic VRAM benchmarks exist for training.

FAQ

What is the minimum VRAM for HunyuanVideo?

45 GB for 544p (544×960×129 frames) per the official model card. With FP8 quantization and CPU offload, users report working on 24 GB cards at reduced resolution.

Source: https://huggingface.co/tencent/HunyuanVideo

Can I run HunyuanVideo on an 8 GB GPU?

Only with the community 8 GB workflow JSON (attached to Civitai 13522) and aggressive offloading. Expect very slow generation and frequent OOM. The ComfyUI blog commenters advise against it.

Source: https://civitai.com/articles/13522; https://blog.comfy.org/p/hunyuanvideo-native-support-in-comfyui (comment by KotatsuAi)

What is the difference between Normal and Master prompt rewrite modes?

Normal mode improves instruction following. Master mode enhances composition, lighting, and camera movement but may drop semantic details. Both are part of the separate Prompt Rewrite model (Hunyuan-Large), not the main diffusion model.

Does HunyuanVideo support image-to-video?

Yes — a separate I2V model exists at github.com/Tencent/HunyuanVideo-I2V with its own checkpoints. The text-to-video model described here is T2V only.

Source: https://github.com/Tencent/HunyuanVideo

Where do I get the FP8 UNET safetensors?

Civitai model 1356617 (pruned FP8) or the official Tencent HuggingFace repo (hunyuan-video-t2v-720p/transformers/mp_rank_00_model_states_fp8.pt — convert to safetensors).

Source: https://civitai.com/articles/13522; https://huggingface.co/tencent/HunyuanVideo

What launch flags should I use for a 24 GB card?

python main.py --fp32-text-enc --fp32-vae --use-sage-attention

No --highvram, no --bf16-unet. Load the FP8 UNET (Civitai 1356617) and BF16 pruned VAE (Civitai 1489354).

Source: https://civitai.com/articles/13522; https://blog.comfy.org/p/hunyuanvideo-native-support-in-comfyui

Sources

• Tencent HunyuanVideo model card — VRAM tables, FP8 release date, architecture details: https://huggingface.co/tencent/HunyuanVideo
• ComfyUI native support announcement — workflow file list, folder structure: https://blog.comfy.org/p/hunyuanvideo-native-support-in-comfyui
• Tencent HunyuanVideo GitHub — inference repo, docker, requirements: https://github.com/Tencent/HunyuanVideo
• Civitai article 13522 — model file recommendations, flags, 8 GB workflow: https://civitai.com/articles/13522
• Civitai article 9798 comments — LoRA training config, Windows DeepSpeed issues, VRAM at 768 px: https://civitai.com/articles/9798/training-a-lora-for-hunyuan-video-on-windows

Internal links

• ComfyUI native workflow deep-dive
• FP8 quantization explained for video models
• LoRA training on Linux with diffusion-pipe

Image placeholders