Train LoRAs on Limited Hardware

Training a LoRA on limited hardware means choosing between three paths: cloud for raw power with time caps, AMD for permanence with setup cost, or local with control bounded strictly by VRAM. This guide maps what each tier realistically supports.

TL;DR

• Free Colab: Use cloud GPUs for a few hours. Use the LoRA dataset guide to prepare data, then train at zero cost.
• AMD ROCm: Train locally on RX 6800 XT and above. Uninstall bitsandbytes and xformers first — the scripts won't launch otherwise.
• Local VRAM tiers: 6 GB for SD1.5 at 512×512. 12 GB unlocks SDXL. 24 GB gives full freedom. Enable --gradient_checkpointing and mixed precision.

What does "limited hardware" mean for LoRA training?

Hardware is limited when VRAM — not compute time — is the binding constraint. The kohya-ss documentation notes that SDXL LoRA training "requires a lot of VRAM" and that "even with 24GB VRAM, you might run out of memory depending on settings" (source). What feels "limited" depends on the base model: SD1.5 trains at 512×512, while SDXL defaults to 1024×1024. For detailed step-by-step instructions, see the kohya-ss training workflow.

Can I train a LoRA for free on Google Colab?

Google Colab gives free access to cloud GPUs (~20 hours/week, community-reported). Use two notebooks: the dataset guide to scrape images and curate tags, then the XL Trainer notebook for kohya-ss training. A community user reported ~10 hours total end-to-end for an SDXL LoRA, with most time spent in dataset prep (source). Colab Pro A100 runs individual training in under 30 minutes. Save every epoch on the free tier — disconnects happen under idle pressure. Use --lowram when RAM is below VRAM.

Did anyone validate training LoRAs on AMD GPUs?

One AMD setup uses Derrian's LoRA Easy Training Scripts on Ubuntu 24.02 with ROCm 6.2 and Python 3.10.14. The author reports a high-quality LoRA in ~40 minutes on an RX 6900XT (70 images, 8 epochs, CAME optimizer) (source). This is competitive with mid-range cloud GPUs, though setup took significantly longer.

Critical: uninstall bitsandbytes and xformers before training. If either is present, the scripts won't launch (source). The CAME optimizer lives in custom_scheduler/LoraEasyCustomOptimizer/came.py (source). Install Python 3.10.14 via pyenv (Ubuntu 24.02 ships 3.12). Use PyTorch with ROCm index: pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.1 (source).

What VRAM tiers exist and what can each train?

The kohya-ss docs don't publish a VRAM-vs-batch-size table, but confirmed parameters from community reports give this:

Each memory-saving flag has a cost: --gradient_checkpointing reduces VRAM at the expense of slower training. --cache_latents disables image augmentations. --cache_text_encoder_outputs eliminates text encoder computation and requires --network_train_unet_only to keep training valid. Every setting is a resource trade-off, not a free optimization.

What settings should I pick for my hardware tier?

Core training parameters follow the same logic across paths. The table consolidates recommended values:

When VRAM is tight, AdamW8bit is the most common optimizer. Prodigy auto-adjusts the learning rate (set around 1.0) but cannot be combined with LoRA+. Adafactor is described as "even more memory-efficient" but requires more careful configuration.

How many steps should I train for?

Community-tested workflows converge on 250–1000 total steps. The formula:

(images × repeats × epochs) ÷ batch_size = total steps

Examples for SDXL: 10 images × 10 repeats × 10 epochs ÷ 2 batch = 500 steps. 100 images × 1 repeat × 10 epochs ÷ 4 batch = 250 steps. SD1.5 usually needs roughly double the repeats to reach comparable step counts (source).

Fewer steps under-train; the LoRA produces no visible effect. Too many steps risk distortion. Styles may need more epochs at lower learning rates than characters. Save every epoch, then pick the best checkpoint by eye.

Common errors and fixes

For systematic diagnosis beyond this table, see the LoRA training debugging workflow and optimizer comparison guide.

Frequently asked questions

How much VRAM do I need to train a LoRA locally? 6 GB is the minimum for SD1.5 at 512×512 with batch size 1 and gradient checkpointing. For SDXL at 1024×1024, budget 12–24 GB depending on batch size and whether you cache latents and text encoder outputs (source).

Can I train LoRAs on an AMD GPU without cloud compute? Yes. Derrian's scripts on ROCm 6.2 with Ubuntu 24.02 validate on RX 6800 XT and above. One user reports ~40 minutes per LoRA on an RX 6900XT (70 images, 8 epochs). Setup requires Python 3.10.14 and uninstalling bitsandbytes/xformers (source).

How many images do I need for a character LoRA? 5 images produce a usable but limited result. 20–50 images with varied poses, angles, and clothing generalize significantly better. Include at least two examples of each pose, angle, and background you want the model to reproduce (source).

What is the difference between undertrained, overtrained, and overfit LoRAs? Undertrained LoRAs ignore the learned concept — typically from a learning rate that is too low or too few steps. Overtrained LoRAs generate distorted artifacts — from a learning rate that is too high. Overfit LoRAs work with only very specific prompts and refuse to generalize — usually from a dataset that is too small or poorly tagged (source).

Should I use LoRA or LoCon for my first run? LoRA works for most cases. LoCon adds more learning layers and may benefit artstyle training specifically. For characters and general-purpose use, start with LoRA (source).

Does network_dim need to be large for good results? No. Higher dim increases capacity but also file size without guaranteed quality improvement. Dim 16 with alpha 8 produces an ~18 MB file and is a widely used baseline (source, kohya-ss).

Why is SDXL training so much more demanding on VRAM? SDXL's larger U-Net architecture and dual text encoders consume substantially more memory than SD1.5. The kohya-ss docs recommend gradient checkpointing, latent caching, and mixed precision specifically to manage this (source).

Sources