Z-Image Turbo: Local Setup & Training

TL;DR

• Run Z-Image Turbo locally with 6B parameters, 8-step inference, sub-second latency on 16 GB VRAM; install via pip install git+https://github.com/huggingface/diffusers and load Tongyi-MAI/Z-Image-Turbo in bfloat16.
• Build lightweight img2img and inpainting workflows in ComfyUI using ImageScaleToMaxDimension, VAE Encode (for inpainting), and Repeat Latent Batch — no ControlNet required; keep denoise between 0.75–0.90 for img2img.
• Train character LoRAs with the ostris zimage_turbo_training_adapter_v2 (324 MB) in AI Toolkit: 3,000 steps, rank 16, lr 1e-4, AdamW8bit, multi-resolution buckets (512/768/1024), single-word captions; ~15–22 min on RTX 5090, ~8–12 h on RTX 4090.

Z-Image Turbo is a 6-billion-parameter distilled model from Tongyi-MAI that generates images in 8 NFEs (number of function evaluations) while matching or exceeding larger competitors on quality benchmarks. It ranks #1 among open-source models on the Artificial Analysis leaderboard and renders bilingual Chinese/English text natively. This guide covers local inference, ComfyUI img2img and inpainting workflows that avoid ControlNet, and a reproducible LoRA training recipe using the ostris de-distillation adapter.

Local inference setup

pip install git+https://github.com/huggingface/diffusers

import torch
from diffusers import ZImagePipeline

pipe = ZImagePipeline.from_pretrained(
    "Tongyi-MAI/Z-Image-Turbo",
    torch_dtype=torch.bfloat16,
).to("cuda")

prompt = "Young woman in red Hanfu, intricate embroidery, neon lightning-bolt lamp above palm, pagoda silhouette at night"

image = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    num_inference_steps=9,
    guidance_scale=0.0,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("zimage_turbo.png")

Key settings: guidance_scale=0.0, num_inference_steps=9 (yields 8 forwards), bfloat16 on CUDA. The model fits in 16 GB VRAM; enable pipe.enable_model_cpu_offload() if you need to run on less.

ComfyUI img2img workflow (no ControlNet)

The Civitai community has converged on a lightweight img2img graph that controls input/output resolution without ControlNet. Start from the default ZiT Text to Image template and rewire as follows.

Nodes to add

Connections

1. Load VAE → VAE input of VAE Encode
2. Load Image → Image input of ImageScaleToMaxDimension
3. ImageScaleToMaxDimension → Pixels input of VAE Encode
4. VAE Encode → Samples input of Repeat Latent Batch
5. Repeat Latent Batch → latent_image input of KSampler

ImageScaleToMaxDimension defaults to 512 max edge; raise it to 1024 or 1536 for higher fidelity. Supported upscale_method values: area, lanczos, bilinear, nearest-exact, bicubic.

Denoise sweet spot

Community testing reports the best img2img results with denoise 0.75–0.90 on the KSampler:

Lower denoise = more source fidelity; higher = more prompt adherence. Start at 0.85 and adjust.

ComfyUI inpainting workflow

Inpainting uses the VAE Encode (for inpainting) node, which accepts both the image and a mask. The same ImageScaleToMaxDimension node can drive resolution.

Nodes to add

Connections

1. Load Image → Pixels + Mask → VAE Encode (for inpainting)
2. Load Image → Pixels → ImageScaleToMaxDimension → image → VAE Encode (for inpainting)
3. Load VAE → vae → VAE Encode (for inpainting)
4. VAE Encode (for inpainting) → Latent → **Repeat Latent Batch → Samples`
5. **Repeat Latent Batch → Latent → **KSampler** → latent_image`

Optional: add the Image Compare node (via ComfyUI Manager) to preview image_a (scaled input) vs image_b (VAE Decode output) side-by-side.

Training a character LoRA with AI Toolkit

Z-Image Turbo is a distilled model; training a LoRA directly on it breaks the 8-step speed ("Turbo Drift"). The ostris zimage_turbo_training_adapter_v2 solves this: it is a 324 MB de-distillation LoRA trained on thousands of Z-Image Turbo outputs at lr 1e-5. You train your LoRA on top of this adapter, then **remove the adapter at inference` — your LoRA retains 8-step speed.

Prerequisites

Install AI Toolkit:

git clone https://github.com/ostris/ai-toolkit.git
cd ai-toolkit
git submodule update --init --recursive
pip install -r requirements.txt

Download the adapter:

huggingface-cli download ostris/zimage_turbo_training_adapter zimage_turbo_training_adapter_v2.safetensors
# Place in your weights folder, e.g. ./weights/zimage_turbo_training_adapter_v2.safetensors

Dataset preparation

• Character LoRA: 70–80 images for high quality (skin texture), 15+ for a working test
• Style LoRA: 30–120 images (~45 sweet spot)
• Resolution: 1024×1024 recommended; 512 works but lower fidelity
• Composition: 40–50% close-ups, 30–40% medium shots, 10–20% full body
• Captions: Single-word trigger per image (e.g., sks or ohwx). Do not write descriptive sentences — the model already knows "clothes," "background," etc. Over-captioning causes "mushy" faces and detail loss.

Folder layout:

datasets/
└── my_character/
    ├── img_001.jpg
    ├── img_001.txt   # contains only: sks
    ├── img_002.jpg
    ├── img_002.txt   # contains only: sks
    └── ...

Training config (YAML)

Save as config/my_zimage_lora.yaml:

job: extension
config:
  name: "my_zimage_lora_v1"
  process:
    - type: "sd_trainer"
      training_folder: "output"
      device: "cuda:0"
      trigger_word: "sks"
      network:
        type: "lora"
        linear: 16
        linear_alpha: 16
      save:
        dtype: "float16"
        save_every: 250
        max_step_saves_to_keep: 4
      datasets:
        - folder_path: "/path/to/datasets/my_character"
          caption_ext: "txt"
          caption_dropout_rate: 0.05
          cache_latents_to_disk: true
          resolution: [512, 768, 1024]
      train:
        batch_size: 1
        steps: 3000
        gradient_accumulation_steps: 1
        train_unet: true
        train_text_encoder: false
        gradient_checkpointing: true
        noise_scheduler: "flowmatch"
        optimizer: "adamw8bit"
        lr: 1e-4
        ema_config:
          use_ema: true
          ema_decay: 0.99
        dtype: "bf16"
      model:
        name_or_path: "Tongyi-MAI/Z-Image-Turbo"
        arch: "zimage"
        assistant_lora_path: "./weights/zimage_turbo_training_adapter_v2.safetensors"
        quantize: true
      sample:
        sampler: "flowmatch"
        sample_every: 250
        width: 1024
        height: 1024
        prompts:
          - "sks, portrait photo, natural lighting"
        seed: 42
        guidance_scale: 1
        sample_steps: 8

Apple Silicon (MPS) adjustments: change device: "mps:0", optimizer: "adamw" (adamw8bit is CUDA-only), quantize: false, add num_workers: 0 under datasets.

Run training

cd ai-toolkit
python run.py config/my_zimage_lora.yaml

Expected wall-clock times (3,000 steps):

Monitor loss decay and sample images every 250 steps. Stop early if identity stabilizes; push to 5,000 steps only with clean data and captions.

Inference with the trained LoRA

ComfyUI: place my_zimage_lora_v1.safetensors in ComfyUI/models/loras/, add a Load LoRA node between the model loader and KSampler, set strength 0.8–1.0.

Python (Diffusers):

from diffusers import ZImagePipeline
import torch

pipe = ZImagePipeline.from_pretrained(
    "Tongyi-MAI/Z-Image-Turbo",
    torch_dtype=torch.bfloat16,
).to("cuda")

pipe.load_lora_weights("./output/my_zimage_lora_v1.safetensors", adapter_name="character")
pipe.set_adapters(["character"], adapter_weights=[0.9])

image = pipe(
    prompt="sks, portrait photo, natural lighting, 85mm lens",
    num_inference_steps=9,
    guidance_scale=0.0,
    width=1024,
    height=1024,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("lora_result.png")

Critical: do not load the training adapter at inference. Your LoRA alone preserves the distilled 8-step speed.

Sources

Common errors and fixes

FAQ

How many images do I need for a character LoRA? 15+ images and 1,000+ steps for a working test; 70–80 images at 1024×1024 for production quality that reproduces skin texture.

What denoise value should I use for img2img? Start at 0.85. Below 0.75 keeps the source nearly unchanged; above 0.90 drifts toward pure text-to-image.

Do I need ControlNet for img2img or inpainting? No. The lightweight workflows above use only native ComfyUI nodes (ImageScaleToMaxDimension, VAE Encode, Repeat Latent Batch) and match ControlNet-style control for most use cases.

Why does my LoRA training produce artifacts after 4,000+ steps? The training adapter's de-distillation effect wears off over long runs. Stay at 3,000 steps unless you have pristine data and captions; beyond 5,000 steps distillation breakdown causes artifacts.

Can I train on Apple Silicon? Yes. Use device: mps:0, optimizer: adamw, quantize: false, num_workers: 0. Expect ~21 hours for 3,000 steps on M4 Pro 64 GB.

What resolution should I train at? Multi-resolution buckets [512, 768, 1024] with 1024×1024 as the target. The 512 bucket is the training bucket size, not a downscale of your dataset.

Is fp32 (full precision) required? Community reports fp32 saves eliminate hallucinations and boost quality at 5,000+ steps. If VRAM allows, keep quantize: false and save in fp32; otherwise bf16 with quantization is the practical default.

How do I remove the training adapter at inference? Simply don't load it. Your trained LoRA is the only extra weight you load on top of the base Z-Image Turbo model.

Related guides

• ComfyUI basics for diffusion models
• LoRA training fundamentals
• Z-Image vs Flux comparison
• RunPod GPU rental cheat sheet
• VAE selection for diffusion models