How many images do I need to train a LoRA?

The community baseline is 20 images minimum, with varied angles, distances, and lighting. More images improve generalization, but 20 is the established starting point.

What learning rate should I use for my first LoRA?

Start with 1e-4 for the UNet and 1e-5 for the text encoder. If training stalls, increase the UNet rate toward 2e-4; if unstable, reduce to 5e-5.

Can I train a LoRA on Google Colab for free?

Yes. hollowstrawberry's Lora Trainer runs on free T4 Colab GPUs. T4 VRAM is limited, so SDXL works but with constraints on batch size and resolution.

What is the difference between LoRA and LoCon?

LoRA is the standard low-rank adaptation format. LoCon (LoRA beyond Conventional) is more precise for style LoRAs but requires a separate extension to load in the WebUI.

Train Your First LoRA Free: Data, Captions, Settings

Free LoRA training guide: dataset prep, WD14/BLIP captioning, and the hyperparameters that actually matter. Colab + local workflows.

Updated 2026-07-25 8 sources cited Training LoRAs

On this page

TL;DR
Introduction
Phase 1: Dataset Preparation
Phase 2: Captioning
Phase 3: Training Settings
Free training environments
Common errors and fixes
FAQ
What remains unverified
Sources

Train Your First LoRA Free: Data, Captions, Settings

TL;DR

Gather 20+ images of your subject with varied angles, distances, and lighting.
Automate tagging (WD14 Tagger) for keyword-based training or captioning (BLIP) for natural-language prompts — both available in hollowstrawberry's free Colab notebooks.
Set learning rates at 1e-4 for the UNet and 1e-5 for text encoders; train for 8–12 epochs; use SDXL memory savers like --gradient_checkpointing and --cache_latents.
Start free with hollowstrawberry's Colab Lora Trainer, or run kohya-ss/sd-scripts locally on your own GPU.

Introduction

Low-Rank Adaptation (LoRA) fine-tunes diffusion models by injecting small, trainable weight matrices into existing layers while keeping the original weights frozen. The result is a portable file you swap between projects to inject a specific face, style, or concept. LoRA's low computational cost means consumer GPUs or free Colab notebooks work; kohya-ss provides a complete local pipeline.

This guide walks through the three phases that determine success: preparing your dataset, captioning it correctly, and dialing in the training settings.

Phase 1: Dataset Preparation

A LoRA trained on a sloppy dataset produces a sloppy model. The community rule of thumb: at least 20 images, with variety in angle, distance, and context.

What to include

For character LoRAs: face closeups, full-body shots, cropped context images. For style LoRAs: vary composition, keep style consistent. Trigger‑word LoRAs need the concept from multiple viewpoints.

The community recommends varied distances from the target, explicitly calling out face closeups alongside full‑body images so the model learns the subject at multiple scales. DreamBooth‑style training (class tokens, not captions) benefits from a focused set centered on one subject (Civitai 7).

Where to collect images

Images: Pixiv, Danbooru, Safebooru, Google, Reddit, own photography. hollowstrawberry's Dataset Maker notebook scrapes Gelbooru, filters tags, deduplicates via FiftyOne — all in‑browser, no local GPU (hollowstrawberry/kohya‑colab).

Organizing files

Single directory, no subfolders. DreamBooth: class token in folder name. Fine‑tuning: JSON metadata per subset.

Phase 2: Captioning

Your captions determine what the LoRA learns. Two automated tools dominate this space, and they serve different training paradigms.

WD14 Tagger: keyword tags for DreamBooth

WD14 Tagger outputs confidence‑scored comma‑separated tags. Standard for DreamBooth‑style training.

Script: finetune/tag_images_by_wd14_tagger.py in kohya‑ss/sd‑scripts. --onnx for speed. Default threshold 0.35. Key flags: --remove_underscore, --character_tags_first, --always_first_tags.

Output: .txt per image, e.g., 1girl, long_hair, blue_eyes, wearing_crown, holding_sword.

BLIP: natural‑language captions for fine‑tuning

BLIP generates sentences: "A girl with long purple hair stands in a garden holding a sword." Suited for fine‑tuning on descriptions, not class tokens.

Manual cleanup

Review outputs. Remove misidentifications, standardize naming (holding_sword not holding_sword, holding weapon), ensure trigger words start tag lines. Use caption_prefix/caption_suffix in .toml for global quality modifiers like masterpiece, best quality,.

Phase 3: Training Settings

Data ready, captioned — now configure.

Command‑line baseline (SD 1.5)

accelerate launch --num_cpu_threads_per_process 1 train_network.py \
 --pretrained_model_name_or_path="<path to SD model>" \
 --dataset_config="my_dataset_config.toml" \
 --output_dir="<output>" --output_name="my_lora" \
 --save_model_as=safetensors --network_module=networks.lora \
 --network_dim=16 --network_alpha=1 --learning_rate=1e-4 \
 --optimizer_type="AdamW8bit" --lr_scheduler="constant" \
 --sdpa --max_train_epochs=10 --save_every_n_epochs=1 \
 --mixed_precision="fp16" --gradient_checkpointing

SDXL: sdxl_train_network.py with network_dim=32, network_alpha=16, split text encoder LRs (--text_encoder_lr1/--text_encoder_lr2), bf16, --cache_text_encoder_outputs + --cache_latents (kohya‑ss sdxl_train_network.md).

Settings that matter

Learning rate — 1e-4 standard. Too low (1e-6) = weak; too high (1e-2) = artifacts. Kohya docs: "relatively higher values (e.g., 1e-4 to 1e-3) often used." Text encoder LR ≈ 1e-5 to prevent overfitting (kohya‑ss train_network.md).

Epochs — 8–12. Below 8 = weak; above 12 = diminishing returns/overfit. num_repeats in .toml multiplies effective dataset size.

network_dim / network_alpha — network_dim range 4–128. Start: 16 (SD 1.5), 32 (SDXL). Higher = more detail, larger file. network_alpha ≈ half network_dim; equal to dim mimics old LoRA.

Optimizer — AdamW8bit (8‑bit via bitsandbytes) halves memory. Alternatives: Lion, DAdaptation, Adafactor (Kohya‑ss docs).

Mixed precision + gradient checkpointing — --mixed_precision="bf16" (SDXL) or "fp16" (SD 1.5) halves VRAM. --gradient_checkpointing trades speed for VRAM. SDXL: --cache_latents + --cache_text_encoder_outputs; text encoder caching forces --network_train_unet_only.

Multi‑LoRA — ComfyUI LoRA Stack (comfyroll). Strength: ~0.6 multi, 0.7–0.8 single. Negative strengths supported.

Free training environments

hollowstrawberry's Colab notebooks

Ready‑to‑run notebooks: Dataset Maker, Lora Trainer, XL Lora Trainer. Free T4 GPUs; XL supports A100 when available. Google Drive persistence.

Dataset Maker: Gelbooru scrape → FiftyOne dedupe → WD14 tag → BLIP caption → one‑click to Lora Trainer.

Civitai 1591 (malcolmrey SDXL guide) notes that SDXL needs more VRAM than most users have locally — cloud Colab or paid GPU (RunPod) is pragmatic. Free T4 remains cheapest entry.

Local: kohya‑ss/sd‑scripts

GitHub repo (Apache 2.0) — LoRA, LoCon/LyCORIS, textual inversion, fine‑tuning. Install with Python 3.10.x and Git. CLI training. Full train_network.py reference in docs.

Common errors and fixes

Symptom	Cause	Fix	Source
LoRA weak/no effect	LR too low or <8 epochs	Increase LR toward 1e‑4 or raise epochs to 8+	rulles (Civitai 60)
Washed‑out images	No VAE active	Activate VAE; check Quicksettings in A1111	rulles (Civitai 60)
UI crashes on LoRA	Corrupted file	Redownload from Civitai; correct base model	rulles (Civitai 60)
OOM during training	MP/GC disabled	Add `--mixed_precision="bf16"`/`"fp16"` + `--gradient_checkpointing`	kohya‑ss train_network.md
Text encoder LoRA frozen	`--cache_text_encoder_outputs` disables it	Remove flag or add `--network_train_unet_only`	kohya‑ss sdxl_train_network.md
Overfit at 1.0 strength	Full power on one or stacked LoRAs	Reduce to ~0.6 multi / ~0.7–0.8 single	rulles (Civitai 60)
Low detail at high res	`network_dim` too low / res too high	Increase `network_dim` 16→32+; lower res	kohya‑ss train_network.md

FAQ

How many images? 20 minimum. Face closeups, full body, varied angles. More helps, 20 is the baseline.

What learning rate? 1e‑4 UNet / 1e‑5 text encoder (kohya‑ss train_network.md). Stalled → 2e‑4; unstable → 5e‑5.

What are network_dim / network_alpha? network_dim 4–128. 16 (SD 1.5), 32 (SDXL). network_alpha ≈ half network_dim.

How to caption? DreamBooth: WD14 Tagger (comma tags). Fine‑tune: BLIP (sentences). Both in hollowstrawberry Colab and kohya‑ss scripts.

Free Colab training? Yes. hollowstrawberry Lora Trainer on free T4. T4 VRAM limited → SDXL works with batch/resolution constraints.

LoRA vs LoCon? LoRA = standard. LoCon = more precise for styles, needs a1111‑sd‑webui‑lycoris to load in WebUI. kohya‑ss trains LoCon natively.

Max LoRA strength? No hard cap. Community: ~0.6 multi, ~0.7–0.8 single. 1.0 risks overfit. From Civitai 60.

How many epochs? 8–12. <8 = weak; >12 = plateau/overfit. One epoch = one full dataset pass.

WD14 tagger usage? Runs finetune/tag_images_by_wd14_tagger.py with --onnx. Docs example: SmilingWolf/wd-swinv2-tagger-v3; default: SmilingWolf/wd-v1-4-convnext-tagger-v2. In hollowstrawberry Colab.

What remains unverified

Exact VRAM requirements for SD1.5 vs SDXL LoRA training — verified sources confirm SDXL needs more VRAM, but specific GPU GB thresholds are not stated.
Concrete pricing/content of RunPod — source 1591 mentions it but does not state costs or minimum specs.
Content of malcolmrey's Civitai model page (article 7) and both Rentry guides (lycoris-from-dreambooth & sdxl-lora-training) — URLs confirmed, page bodies not accessible.
Whether free T4 Colab tier provides sufficient VRAM for every SDXL workflow — hollowstrawberry notebooks support T4, but the SDXL training guide implies paid cloud GPU is pragmatic.
Real‑world error strings from failed first‑training attempts — none appeared in verified sources.

Sources

kohya-ss/sd-scripts — train_network.md, sdxl_train_network.md, config_README-en.md, wd14_tagger_README-en.md
hollowstrawberry/kohya-colab — free Colab notebooks
hollowstrawberry/stable-diffusion-guide — HuggingFace model card
Civitai 7 — malcolmrey DreamBooth/LyCORIS/LoRA guide
Civitai 60 — rulles "Everything I Know About Lora"
Civitai 1591 — malcolmrey SDXL LoRA Training
Civitai 6831 — ThinkDiffusion ComfyUI LoRA guide