ArtificialGuyBR

Home / Blog / Training LoRAs

Textual Inversion Training Guide 2025: Embeddings vs LoRAs

When to use Textual Inversion over LoRA, Kohya_ss and A1111 settings, and the learning-rate gap explained.

10 sources cited Training LoRAs

On this page

  1. TL;DR
  2. When an Embedding Still Beats a LoRA
  3. How Textual Inversion Works
  4. Training in Kohya_ss (SD 1.5 / SDXL)
  5. Training in A1111 WebUI
  6. Using the Trained Embedding
  7. Common Errors and Fixes
  8. FAQ
  9. Model Support Summary
  10. Sources

Textual Inversion Training Guide 2025: Embeddings vs LoRAs

Textual Inversion (TI) teaches Stable Diffusion new concepts by learning only token embeddings inside a frozen text encoder. Unlike LoRA or full fine-tuning, TI never touches the model weights — it simply adds a new "word" to the tokenizer and optimizes its vector representation. The result is a tiny .safetensors file (typically 10–50 KB) that you drop into your embeddings folder and invoke by filename in prompts.

Textual inversion new token → existing weights Learns a vector in token space. Model weights untouched. Kilobytes. Portable inside one base. LoRA low-rank delta → the weights Learns a change to the weights. Alters how the model computes. Megabytes. Tied to its base. Both steer output, at different layers of the stack. Rule of thumb: an embedding can name a thing the model already knows; a LoRA can teach it something new.
Embedding vs LoRA: what each one actually changes

How LoRA training differs from TI — the two methods complement each other for different use cases.

TL;DR

When an Embedding Still Beats a LoRA

Scenario | Why TI wins | Source

|----------|-------------|--------

Cross-model portability | TI trained on vanilla base model works on most finetunes | A1111 wiki VRAM limited | Only token embeddings update; U-Net and VAE stay frozen | Kohya_ss docs Quick iteration | 1,000–2,000 steps typical for TI | Kohya_ss docs Style transfer | --use_style_template isolates aesthetic without subject bias | Kohya_ss docs Embeddings shine when you need a portable concept token that works across models derived from the same base. LoRAs still win for multi-subject scenes and precise pose control.

How Textual Inversion Works

TI adds a special placeholder token to the tokenizer (e.g., <mychar>), expands the embedding matrix by one row, then freezes the entire model — U-Net, VAE, and text encoder — while optimizing only that new row. During training, each image is paired with a prompt template containing the placeholder. The gradient flows through the frozen text encoder into the new embedding vector, nudging it until the model reconstructs the training images when prompted.

The mechanism is model-specific by design. The embedding dimension, tokenizer vocabulary, and text encoder architecture are fixed per model family. SD 1.5 uses a 768-dim CLIP-L encoder; SDXL uses dual CLIP-L/CLIP-G encoders; FLUX and SD3 use entirely different transformer architectures. This is why a TI embedding trained on SD 1.5 cannot load on SDXL or FLUX — the token ID space and embedding dimensions do not match.

Training in Kohya_ss (SD 1.5 / SDXL)

Prerequisites

Core Parameters

Parameter | Recommended Value | Notes

|-----------|-------------------|-------

--pretrained_model_name_or_path | path to v1-5-pruned-emaonly.safetensors or SDXL base | Must be vanilla base model --token_string | unique string like mychar123 | Must not exist in tokenizer --init_word | concept word like woman or cat | Multi-word allowed (e.g., anime girl) --num_vectors_per_token | 1 (default) or 4–5 for complex subjects | More vectors = more prompt tokens consumed --learning_rate | 1e-6 (Kohya docs) or 2e-6 (code default) or 0.005 (paper) | See learning rate note --lr_scheduler | constant | --optimizer_type | AdamW8bit (preferred) or AdamW | AdamW8bit + bf16 may error on Linux --mixed_precision | bf16 (RTX 30/40) or fp16 | --max_train_steps | 1000–2000 | Save checkpoints every 200 steps --cache_latents | enabled | Reduces VRAM significantly --use_object_template | enabled (for subjects) | Ignores caption files --use_style_template | enabled (for styles) | Ignores caption files

Learning Rate Discrepancy

Three values circulate in the community:

All produce working embeddings. The 0.005 value reproduces the paper's setup; 1e-6/2e-6 account for Kohya_ss's different optimizer, precision, and typical dataset sizes. Start at 1e-6, increase only if convergence stalls.

Example Command (SD 1.5)

accelerate launch --num_cpu_threads_per_process 2 train_textual_inversion.py \
  --pretrained_model_name_or_path="models/Stable-diffusion/v1-5-pruned-emaonly.safetensors" \
  --train_data_dir="data/mychar" \
  --output_dir="output/mychar" \
  --output_name="mychar-ti" \
  --save_model_as=safetensors \
  --token_string="mychar123" \
  --init_word="woman" \
  --num_vectors_per_token=5 \
  --learning_rate=1e-6 \
  --lr_scheduler="constant" \
  --optimizer_type="AdamW8bit" \
  --mixed_precision="bf16" \
  --cache_latents \
  --use_object_template \
  --vae="models/VAE/vae-ft-mse-840000-ema-pruned.safetensors" \
  --max_train_steps=1000 \
  --save_every_n_steps=200

For SDXL, use sdxl_train_textual_inversion.py with --resolution=1024,1024 and the SDXL base checkpoint.

Step-by-step LoRA training tutorial — if you decide LoRA fits your use case better.

Training in A1111 WebUI

  1. Open the Textual Inversion tab
  2. Create embedding: name it, set initialization text (e.g., woman), choose vectors per token (1–16)
  3. Preprocess: point to source images, enable "Create flipped copies" for data augmentation
  4. Train: select embedding, set learning rate (default 0.005), max steps (1,000–5,000), log directory
  5. Template: use subject.txt for objects/people, style.txt for aesthetics
  6. Start training — monitor loss and preview images

A1111 supports learning rate scheduling via prompt syntax: 0.005:100, 1e-3:1000, 1e-5 drops LR at specified steps.

Using the Trained Embedding

  1. Copy the .safetensors file to stable-diffusion-webui/embeddings/
  2. No restart needed — the embedding loads immediately
  3. In prompts, use the filename (without extension) as the token: photo of mychar-ti
  4. Weight it like any token: (mychar-ti:1.2) increases influence

Negative embeddings and token weighting guide — pair your trained embedding with proper weighting for best results.

Common Errors and Fixes

Symptom | Cause | Fix

|---------|-------|-----

Loss: nan | Learning rate too high | Drop to 1e-6 or lower; use constant scheduler Token string already exists in tokenizer | Placeholder collides with vocabulary | Add numbers/special chars: mychar123, <mychar> CUDA setup error with AdamW8bit + bf16 | Linux platform incompatibility | Switch to AdamW + bf16 or AdamW8bit + fp16 Embedding works on training model only | TI is model-specific by design | Train on vanilla base model for best portability No improvement after 500+ steps | Token string missing from captions/templates | Ensure --use_object_template enabled or token appears in captions Overtraining artifacts at 1,300+ steps | Too many steps for dataset size | Stop at 800–1,000 steps; use earlier checkpoint

FAQ

When should I use an embedding instead of a LoRA? Use embeddings when file size matters (sharing, mobile), you need cross-model portability, or you are training a single subject/style with limited VRAM. LoRAs handle multi-concept scenes and higher fidelity better.

What learning rate is best for textual inversion? Start at 1e-6 with Kohya_ss (AdamW8bit, fp16/bf16). Use 0.005 only if reproducing the original paper setup (vanilla Adam, fp32, 3–5 images). Both converge; 1e-6 is safer for larger datasets.

Can I use embeddings across different Stable Diffusion models? Only within the same architecture family. An SD 1.5 embedding works on most SD 1.5 finetunes but not on SDXL, SD3, or FLUX. Each model family needs its own TI training.

How many training images do I need for textual inversion? The original paper showed 3–5 images suffice. Community practice uses 15–25 for robustness. More images require proportionally more steps.

Does textual inversion work with SDXL or FLUX? SDXL: yes, via sdxl_train_textual_inversion.py. FLUX and SD3: no — their transformer architectures lack TI implementations in Kohya_ss or diffusers.

Why is my embedding not working on other models? TI binds to a specific model's tokenizer and text encoder. Finetunes that modify the text encoder (e.g., CLIP skip, merged models) break compatibility. Train on the vanilla base for maximum portability.

What does --num_vectors_per_token do? Each vector consumes one token from the 77-token prompt limit. Default is 1. Values 4–5 capture complex subjects better but reduce prompt space. The paper found a single vector often suffices.

Model Support Summary

Model Family | TI Support | Script SD 1.x / 2.x | Yes | train_textual_inversion.py SDXL | Yes | sdxl_train_textual_inversion.py SD 3 / 3.5 | No | LoRA only FLUX.1 | No | LoRA only HunyuanImage | No | LoRA only Anima | No | LoRA only For SD3, FLUX, and newer architectures, LoRA remains the only personalization option in Kohya_ss.

Sources