Z-Image Turbo LoRA: Dataset, Precision, Adapter, AI Toolkit
Train Z-Image Turbo LoRAs with AI Toolkit: dataset sizing, fp8 quantization, training adapter, sigmoid timestep, and Flux/SDXL differences.
On this page
- TL;DR
- What is Z-Image Turbo and why does training differ?
- Hardware and precision: what VRAM do you actually need?
- The training adapter: v1 vs v2
- Dataset: size, captions, and diversity
- AI Toolkit settings that actually matter
- What differs from Flux and SDXL training
- Common errors and fixes
- FAQ
- Sources
- Further reading
Train a Z-Image Turbo LoRA
TL;DR
- Dataset: 9–20 diverse images beat larger uniform sets; captions optional — tags work with "no caveats."
- Precision: fp8 (qfloat8) quantization cuts VRAM ~50% vs bf16; 16 GB cards can train with
low_vram: true, 24 GB+ needed for unquantized. - Adapter: You must load the Ostris de-distillation adapter (
ostris/zimage_turbo_training_adapter) or the LoRA will lose its 8-step speed. - Settings that differ from Flux/SDXL:
arch: zimage:turbo, sigmoid timestep (weighted), no negative prompts, natural-language prompts preferred.
What is Z-Image Turbo and why does training differ?
Z-Image Turbo is a 6-billion-parameter single-stream diffusion transformer (S3-DiT) from Tongyi Lab (Alibaba) distilled to 8 inference steps via Decoupled-DMD [^1]. The base Z-Image model supports classifier-free guidance (CFG), negative prompts, and 28–50 steps; Turbo drops all three for speed [^2].
Because the model is already distilled, standard LoRA fine-tuning breaks the distillation: the model "reverts toward requiring more inference steps" [^3]. The fix is a de-distillation training adapter — a pre-trained LoRA that has already learned to undo the distillation. You train your LoRA on top of this adapter, then discard it at inference.
Hardware and precision: what VRAM do you actually need?
| Configuration | VRAM (practical) | Notes | Source |
|---|---|---|---|
| Unquantized (NONE) | 24 GB+ | "If you have over 24GB of VRAM you should be good to go with 'NONE'" [^4] | [^4] |
| fp8 / qfloat8 (default) | 16–20 GB | quantize: true, qtype: qfloat8, dtype: bf16 cuts VRAM ~50% [^5] | [^5] |
| fp8 + low_vram flag | 12–16 GB | low_vram: true enables CPU offload for activations [^5] | [^5] |
The official inference card states Turbo "fits comfortably within 16G VRAM consumer devices" [^1]. Training is heavier; community configs default to fp8 for all GPU tiers [^5]. One guide claims "float8 would be slower" [^4]; configs use qtype: qfloat8 with quantize: true as default for all GPU tiers [^5].
Key distinction from Flux/SDXL: Those models typically train in bf16 or fp16. Z-Image Turbo configs default to fp8 quantization for both transformer and text encoder (quantize_te: true, qtype_te: qfloat8) because the distillation is sensitive to precision drift.
The training adapter: v1 vs v2
The Ostris repository hosts ostris/zimage_turbo_training_adapter with two files [^3]:
zimage_turbo_training_adapter_v1.safetensors— stable, widely usedzimage_turbo_training_adapter_v2.safetensors— "in testing" per one guide [^6]; no public changelog exists
In AI Toolkit you reference it via assistant_lora_path: ostris/zimage_turbo_training_adapter/zimage_turbo_training_adapter_v1.safetensors [^5]. If you omit the adapter, the distillation breaks down "very quickly" and "in an unpredictable way" [^3].
Dataset: size, captions, and diversity
| Factor | Recommendation | Evidence | Source |
|---|---|---|---|
| Image count | 9–20 | "9 images (variety in angles and outfits helps)" and "20 is more than fine" [^6][^7] | [^6][^7] |
| Captions | Optional | "Yes, you can train ZIT Loras on tags instead of captions. No caveats. It works." [^8] | [^8] |
Natural-language captions may yield better prompt adherence; one guide notes "Use long, detailed, coherent natural language descriptions... The model responds much better to full sentences and paragraphs" [^8]. For local captioning without upload limits, the ComfyUI JoyCaption extension supports siglip-so400m-patch14-384, Meta-Llama-3.1-8B-bnb-4bit, and joy-caption-pre-alpha [^9].
AI Toolkit settings that actually matter
These are the non-default parameters that community configs and guides agree on [^5][^7][^6]:
| Setting | Value | Why it differs from Flux/SDXL |
|---|---|---|
arch | zimage:turbo | Model-specific architecture key |
assistant_lora_path | ostris/zimage_turbo_training_adapter/...v1.safetensors | Required to preserve 8-step distillation |
timestep_type | weighted (sigmoid) | Distilled models concentrate signal at specific timesteps |
cache_text_embeddings | true | Speeds up repeated forward passes |
lr | 0.0001 (1e-4) | "Increasing it could cause it to crash" [^7] |
steps | 3000 | "Landed around ~1 hour on a 5090" [^6]; concept emerges at 1000–1500 [^6] |
resolution | [512, 768, 1024] | Multi-resolution buckets; default UI is 1024×1024 [^6] |
linear / linear_alpha | 16 or 32 (equal) | LoRA rank; no systematic comparison published |
quantize / qtype | true / qfloat8 | fp8 for memory; dtype stays bf16 |
sample_every | 250 | Preview interval |
sample_lora_weight | 0.7–1.0 | "To see how strong it hits" [^6] |
Trigger word: Must appear in every sample prompt or "early samples look like the base model and it's hard to tell what's working" [^6].
What differs from Flux and SDXL training
| Aspect | Flux / SDXL | Z-Image Turbo |
|---|---|---|
| Distillation | None (full-step) | 8-step Decoupled-DMD |
| Training adapter | Not needed | Required (de-distill) |
| Timestep sampling | Uniform / logit-normal | Sigmoid (weighted) |
| Negative prompts | Supported | Not supported — guidance_scale=0.0 [^2] |
| CFG | Used | Not used |
| Prompt style | Tag soup works | Natural language preferred [^8] |
| Quantization default | bf16 / fp16 | fp8 (qfloat8) for both transformer + TE |
| VRAM (training, 24 GB card) | Comfortable in bf16 | fp8 recommended; "Illustrious costs way less VRAM than ZIT" [^8] |
One guide claims "Train it the same way as Illustrious" [^8], but the config differences above are structural — not optional.
Common errors and fixes
| Symptom | Cause | Fix | Source |
|---|---|---|---|
| Training crashes / OOM | LR too high or VRAM exceeded | Keep LR at 1e-4; enable fp8 (quantize: true, qtype: qfloat8); add low_vram: true | [^7] |
| Early samples = base model | Trigger word missing from prompts | Include trigger word in every sample prompt | [^6] |
| Concept not learned by 2000 steps | Dataset too uniform | Increase pose/angle/outfit variety | [^6] |
| Weird background artifacts | Overfitting or distillation breakdown | Reduce steps; verify adapter loaded; try LoRA weight 0.7–0.8 | [^6] |
| LoRA needs >8 steps at inference | Distillation broke (long training without adapter) | Retrain with adapter; stop at 3000 steps | [^3] |
FAQ
How much VRAM do I need to train a Z-Image Turbo LoRA?
With fp8 quantization (quantize: true, qtype: qfloat8) and low_vram: true, 12–16 GB works. Unquantized training needs 24 GB+. The official inference requirement is 16 GB [^1].
What is the Z-Image Turbo training adapter and why do I need it?
It's a pre-trained LoRA (ostris/zimage_turbo_training_adapter) that "de-distills" the model. Training your LoRA on top of it preserves the 8-step speed. Without it, "the distillation breaks down very quickly" [^3].
Can I train Z-Image Turbo on tags instead of captions?
Yes. "No caveats. It works." [^8] Natural-language captions may yield better prompt adherence; one guide notes "Use long, detailed, coherent natural language descriptions... The model responds much better to full sentences and paragraphs" [^8].
Why does my Z-Image Turbo LoRA need more inference steps after training?
The distillation broke. This happens when training too long without the adapter, or with a high learning rate. Retrain with the adapter and cap at 3000 steps [^3][^6].
What's the difference between training adapter v1 and v2?
v1 is stable; v2 is "in testing" with no public changelog [^6]. Use v1 unless you're experimenting.
Do negative prompts work with Z-Image Turbo?
No. The model card explicitly lists "Negative Prompting: ❌" for Turbo [^2]. Do not include them in prompts or sample configs.
How many training steps should I use?
3000 is the community standard (~1 hour on RTX 5090) [^6]. The concept typically emerges at 1000–1500 steps [^6]. No published data on diminishing returns beyond 3000.
Can I use the RunPod Ostris template for training?
Yes. The official AI Toolkit template is available on RunPod [^6]. Select it, choose your GPU (RTX 5090 used in published runs), and deploy.
Sources
- Z-Image Turbo model card — architecture, distillation, VRAM, CFG/negative prompt support — https://huggingface.co/Tongyi-MAI/Z-Image-Turbo
- Z-Image base model card — comparison table (CFG, steps, negative prompting, diversity) — https://huggingface.co/Tongyi-MAI/Z-Image
- Ostris training adapter card — de-distillation mechanism, distillation breakdown warning — https://huggingface.co/ostris/zimage_turbo_training_adapter
- Ostris AI Toolkit repo — supported models list includes Z-Image Turbo and base — https://github.com/ostris/ai-toolkit
- CitronLegacy config generator — full YAML with fp8, sigmoid timestep, adapter path, resolution buckets — https://github.com/citronlegacy/ai-toolkit-zimage-config-generator/blob/main/example_configs/zimage_AnubisCitronOC_ZIT_v3.yaml
- Civitai: "Here's how to Quickly Train a Z-Image Turbo LoRA" — 9-image test, 3000 steps, 1 hr on 5090, adapter v1/v2, trigger word, sample settings — https://civitai.com/articles/23233/heres-how-to-quickly-train-a-z-image-turbo-lora
- Civitai: "Z Image Turbo - Character LoRA Training Guide" — 20-image dataset, 19 neutral prompts, sigmoid timestep, cache embeddings, LR warning, 24 GB VRAM note — https://civitai.com/articles/23842/z-image-turbo-character-lora-training-guide-easy-how-to-guide
- Civitai: "Z Image Turbo (ZIT) Anime Lora Training Experience" — tags vs captions, natural language prompting, Illustrious VRAM comparison, small-dataset failures — https://civitai.com/articles/24472/z-image-turbo-zit-anime-lora-training-experience
- Civitai: "Z-Image Turbo LoRA training with AI Toolkit and Z-Image ControlNet" — ControlNet Union video chapters, FP8 scaled model 5.7 GB — https://civitai.com/articles/23314/z-image-turbo-lora-training-with-ai-toolkit-and-z-image-controlnet-full-tutorial-for-highest-quality
- arXiv:2511.22699 — Z-Image paper, 314K H800 GPU hours training cost, Decoupled-DMD distillation — https://arxiv.org/abs/2511.22699
- ComfyUI JoyCaption guide — local captioning models for NSFW/private datasets — https://www.runcomfy.com/comfyui-nodes/Comfyui_CXH_joy_caption