Textual Inversion Training Guide 2025: Embeddings vs LoRAs

Textual Inversion (TI) teaches Stable Diffusion new concepts by learning only token embeddings inside a frozen text encoder. Unlike LoRA or full fine-tuning, TI never touches the model weights — it simply adds a new "word" to the tokenizer and optimizes its vector representation. The result is a tiny .safetensors file (typically 10–50 KB) that you drop into your embeddings folder and invoke by filename in prompts.

How LoRA training differs from TI — the two methods complement each other for different use cases.

TL;DR

• Use embeddings for: single subjects, styles, or objects where file size and cross-model portability matter most
• Use LoRAs for: complex characters, multi-concept compositions, or when you need higher fidelity
• Train on vanilla SD 1.5 or SDXL base models — embeddings trained on finetunes rarely transfer well
• Kohya_ss supports TI for SD 1.x/2.x and SDXL only — no TI support exists for FLUX or SD3

When an Embedding Still Beats a LoRA

Embeddings shine when you need a portable concept token that works across models derived from the same base. LoRAs still win for multi-subject scenes and precise pose control.

How Textual Inversion Works

TI adds a special placeholder token to the tokenizer (e.g., <mychar>), expands the embedding matrix by one row, then freezes the entire model — U-Net, VAE, and text encoder — while optimizing only that new row. During training, each image is paired with a prompt template containing the placeholder. The gradient flows through the frozen text encoder into the new embedding vector, nudging it until the model reconstructs the training images when prompted.

The mechanism is model-specific by design. The embedding dimension, tokenizer vocabulary, and text encoder architecture are fixed per model family. SD 1.5 uses a 768-dim CLIP-L encoder; SDXL uses dual CLIP-L/CLIP-G encoders; FLUX and SD3 use entirely different transformer architectures. This is why a TI embedding trained on SD 1.5 cannot load on SDXL or FLUX — the token ID space and embedding dimensions do not match.

Training in Kohya_ss (SD 1.5 / SDXL)

Prerequisites

• Kohya_ss sd-scripts repo cloned and environment set up
• Dataset prepared (15–25 square images at 512×512 for SD 1.5, 1024×1024 for SDXL)
• Vanilla base model checkpoint (not a finetune)

Core Parameters

Learning Rate Discrepancy

Three values circulate in the community:

• 0.005 — used in the original Gal et al. 2022 paper with vanilla Adam on fp32, 3–5 images
• 1e-6 — Kohya_ss documentation recommendation for AdamW8bit, fp16/bf16, larger datasets
• 2e-6 — Kohya_ss code default (--learning_rate default in library/args.py)

All produce working embeddings. The 0.005 value reproduces the paper's setup; 1e-6/2e-6 account for Kohya_ss's different optimizer, precision, and typical dataset sizes. Start at 1e-6, increase only if convergence stalls.

Example Command (SD 1.5)

accelerate launch --num_cpu_threads_per_process 2 train_textual_inversion.py \
  --pretrained_model_name_or_path="models/Stable-diffusion/v1-5-pruned-emaonly.safetensors" \
  --train_data_dir="data/mychar" \
  --output_dir="output/mychar" \
  --output_name="mychar-ti" \
  --save_model_as=safetensors \
  --token_string="mychar123" \
  --init_word="woman" \
  --num_vectors_per_token=5 \
  --learning_rate=1e-6 \
  --lr_scheduler="constant" \
  --optimizer_type="AdamW8bit" \
  --mixed_precision="bf16" \
  --cache_latents \
  --use_object_template \
  --vae="models/VAE/vae-ft-mse-840000-ema-pruned.safetensors" \
  --max_train_steps=1000 \
  --save_every_n_steps=200

For SDXL, use sdxl_train_textual_inversion.py with --resolution=1024,1024 and the SDXL base checkpoint.

Step-by-step LoRA training tutorial — if you decide LoRA fits your use case better.

Training in A1111 WebUI

1. Open the Textual Inversion tab
2. Create embedding: name it, set initialization text (e.g., woman), choose vectors per token (1–16)
3. Preprocess: point to source images, enable "Create flipped copies" for data augmentation
4. Train: select embedding, set learning rate (default 0.005), max steps (1,000–5,000), log directory
5. Template: use subject.txt for objects/people, style.txt for aesthetics
6. Start training — monitor loss and preview images

A1111 supports learning rate scheduling via prompt syntax: 0.005:100, 1e-3:1000, 1e-5 drops LR at specified steps.

Using the Trained Embedding

1. Copy the .safetensors file to stable-diffusion-webui/embeddings/
2. No restart needed — the embedding loads immediately
3. In prompts, use the filename (without extension) as the token: photo of mychar-ti
4. Weight it like any token: (mychar-ti:1.2) increases influence

Negative embeddings and token weighting guide — pair your trained embedding with proper weighting for best results.

Common Errors and Fixes

FAQ

When should I use an embedding instead of a LoRA? Use embeddings when file size matters (sharing, mobile), you need cross-model portability, or you are training a single subject/style with limited VRAM. LoRAs handle multi-concept scenes and higher fidelity better.

What learning rate is best for textual inversion? Start at 1e-6 with Kohya_ss (AdamW8bit, fp16/bf16). Use 0.005 only if reproducing the original paper setup (vanilla Adam, fp32, 3–5 images). Both converge; 1e-6 is safer for larger datasets.

Can I use embeddings across different Stable Diffusion models? Only within the same architecture family. An SD 1.5 embedding works on most SD 1.5 finetunes but not on SDXL, SD3, or FLUX. Each model family needs its own TI training.

How many training images do I need for textual inversion? The original paper showed 3–5 images suffice. Community practice uses 15–25 for robustness. More images require proportionally more steps.

Does textual inversion work with SDXL or FLUX? SDXL: yes, via sdxl_train_textual_inversion.py. FLUX and SD3: no — their transformer architectures lack TI implementations in Kohya_ss or diffusers.

Why is my embedding not working on other models? TI binds to a specific model's tokenizer and text encoder. Finetunes that modify the text encoder (e.g., CLIP skip, merged models) break compatibility. Train on the vanilla base for maximum portability.

What does --num_vectors_per_token do? Each vector consumes one token from the 77-token prompt limit. Default is 1. Values 4–5 capture complex subjects better but reduce prompt space. The paper found a single vector often suffices.

Model Support Summary

| Model Family | TI Support | Script |

| SD 1.x / 2.x | Yes | train_textual_inversion.py |

| SDXL | Yes | sdxl_train_textual_inversion.py |

| SD 3 / 3.5 | No | LoRA only |

| FLUX.1 | No | LoRA only |

| HunyuanImage | No | LoRA only |

| Anima | No | LoRA only |

For SD3, FLUX, and newer architectures, LoRA remains the only personalization option in Kohya_ss.

Sources

• Kohya_ss Textual Inversion documentation — https://github.com/kohya-ss/sd-scripts/blob/main/docs/train_textual_inversion.md
• Kohya_ss train_textual_inversion.py source — https://github.com/kohya-ss/sd-scripts/blob/main/train_textual_inversion.py
• Kohya_ss shared args module — https://github.com/kohya-ss/sd-scripts/blob/main/library/args.py
• A1111 WebUI Textual Inversion wiki — https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Textual-Inversion
• Gal et al. 2022 "An Image is Worth One Word" — https://arxiv.org/abs/2208.01618
• Desi_Cafe Kohya_ss Dreambooth TI tutorial — https://civitai.com/articles/618
• Malcolm Rey Textual Inversion training guide — https://civitai.com/articles/3114
• HuggingFace diffusers Textual Inversion training — https://huggingface.co/docs/diffusers/en/training/text_inversion
• Kohya_sd-scripts FLUX docs (no TI support) — https://github.com/kohya-ss/sd-scripts/blob/main/docs/flux_train_network.md
• Kohya_sd-scripts LoRA training guide (shared args reference) — https://github.com/kohya-ss/sd-scripts/blob/main/docs/train_network.md

json
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {"@type": "Question", "name": "When should I use an embedding instead of a LoRA?", "acceptedAnswer": {"@type": "Answer", "text": "Use embeddings when file size matters, you need cross-model portability, or you are training a single subject/style with limited VRAM."}},
    {"@type": "Question", "name": "What learning rate is best for textual inversion?", "acceptedAnswer": {"@type": "Answer", "text": "Start at 1e-6 with Kohya_ss or 0.005 to reproduce the original paper setup."}},
    {"@type": "Question", "name": "Can I use embeddings across different Stable Diffusion models?", "acceptedAnswer": {"@type": "Answer", "text": "Only within the same architecture family. SD 1.5 embeddings do not work on SDXL, SD3, or FLUX."}},
    {"@type": "Question", "name": "How many training images do I need?", "acceptedAnswer": {"@type": "Answer", "text": "The original paper showed 3–5 images suffice; community practice uses 15–25 for robustness."}},
    {"@type": "Question", "name": "Does textual inversion work with SDXL or FLUX?", "acceptedAnswer": {"@type": "Answer", "text": "SDXL: yes via sdxl_train_textual_inversion.py. FLUX and SD3: no."}},
    {"@type": "Question", "name": "Why is my embedding not working on other models?", "acceptedAnswer": {"@type": "Answer", "text": "TI binds to a specific model tokenizer and text encoder. Finetunes that modify the text encoder break compatibility."}},
    {"@type": "Question", "name": "What does --num_vectors_per_token do?", "acceptedAnswer": {"@type": "Answer", "text": "Each vector consumes one token from the 77-token prompt limit. Default is 1."}}
  ]
}