SDXL LoRA Training: Colab vs Hosted Trainers

TL;DR

• Free T4 GPU: Both Colab and Kaggle offer NVIDIA T4 (16 GB VRAM). Kaggle provides 2× T4 for dual-GPU training; Colab gives 1× T4.
• Session limits: Colab disconnects after 90 minutes idle and caps at 12 hours. Kaggle interactive sessions idle out after 20 minutes and max run time is 12 hours.
• Persistence: Colab uses Google Drive mounts (survives VM termination). Kaggle saves /kaggle/working (20 GB) as notebook output; chain notebooks to reuse files.
• SDXL on T4: Use --full_fp16 to fit optimizer states in VRAM; add --cache_latents --cache_latents_to_disk for major speedup.
• Hosted alternatives: Civitai, Tensor.Art, Replicate, and Hugging Face AutoTrain offer web-based LoRA training with pay-per-second pricing.

For more on Kaggle notebook persistence, see Kaggle notebook chaining guide.

Colab vs. Kaggle: GPU Hardware and Quotas

Both platforms ground users in NVIDIA T4 GPUs, but Kaggle's "T4 x2" accelerator provisions two physical T4 cards in a single container, giving you 29 GB RAM and the ability to run multi-GPU training. Colab's free tier provides a single T4. Neither platform guarantees specific GPU models—Colab's FAQ states "GPU types vary" depending on availability.

Phone verification required: Kaggle mandates phone number verification to unlock GPU access. Unverified accounts receive CPU-only sessions.

Session Timeouts and Persistence

Colab: Drive Mount Over File System

Colab’s runtime terminates after 90 minutes of inactivity, but your files survive through Google Drive. Mount your Drive at /content/drive/MyDrive/ and store checkpoints there:

from google.colab import drive
drive.mount('/content/drive')

The next session re-mounts the same path. This pattern works for any storage: LoRAs, training data, or model weights.

Kaggle: Working Directory vs. Temp Storage

Kaggle notebooks offer two storage zones:

• /kaggle/working — 20 GB auto-saved. Files here become a versioned output; attach them as inputs to future notebooks.
• /kaggle/temp — up to 75 GB scratch space, wiped on session end.

Download large models to /kaggle/temp/models/ for space, but always copy final LoRAs to /kaggle/working/ before the session expires.

Chaining Notebooks for Continuity

Kaggle’s “chain notebooks” feature lets you attach any notebook’s /kaggle/working output as a dataset input. This is the only way to preserve large files across sessions. Save your LoRA checkpoint, then create a new notebook that imports it as data.

SDXL Training Settings for Limited VRAM

For detailed hyperparameter tuning, see SDXL training guide.

Why --full_fp16 Is Necessary

SDXL’s UNet has ~2.6 billion parameters. In fp32, weights alone need 10.4 GB. Add activations and AdamW optimizer states (8 bytes/param) and you exceed 16 GB VRAM.

--full_fp16 stores optimizer states in fp16 (2 bytes/param), cutting peak memory roughly in half. The Kohya trainer notebook enables this automatically for T4 sessions.

Caching Latents to Disk

The --cache_latents --cache_latents_to_disk flags precompute and store latent embeddings on disk instead of recomputing them each training step. The notebook notes this provides a “huge speed up.”

Use both together on T4 to avoid OOM while maintaining reasonable speed.

The --lowram Patch

When the notebook detects a T4 GPU, it patches library/model_util.py to change cuda→cpu during model loading. This enables CPU offload for the initial load, preventing OOM before training even starts.

LoRA Types: What the Trainer Actually Supports

The Kohya trainer’s network_category dropdown (v14.6) offers four options:

Additional types (IA³, LoKR, DyLoRA-Lycoris) exist in the Kohya ecosystem, but require manual network_module + network_args configuration and are not in the UI dropdown.

Hosted Trainers: Web-Based Alternatives

When local cloud sessions timeout, consider hosted trainers that manage compute for you.

For architecture deep-dives, read LoRA implementation details.

Civitai On-Site Training

Civitai provides a built-in LoRA trainer. From any model page, click the “Train” button to launch a training job on Civitai’s GPU pool. You pay with compute credits.

Tensor.Art Online Training

Tensor.Art offers free daily quota for LoRA/LoCon training. Upload your dataset, set parameters, and the platform trains on dedicated GPUs. Results download automatically.

Replicate API

The ostris/sdxl-lora-trainer model on Replicate lets you train via API. Pay per second of GPU usage (A100/H100). Ideal for automation or integration into pipelines.

Hugging Face AutoTrain

AutoTrain Advanced supports SDXL LoRA training on HF’s GPU clusters. Point it at your dataset, choose hyperparameters, and let HF handle the compute.

Common Errors and Fixes

Frequently Asked Questions

Why does my Colab disconnect after 90 minutes? Colab’s free tier kills idle sessions after 90 minutes. Enable “Keep alive” in the menu or use a browser extension to ping the runtime. For longer jobs, break training into smaller chunks and save checkpoints frequently.

How do I keep my LoRA checkpoints after a Kaggle session ends? Save checkpoints to /kaggle/working/ instead of /kaggle/temp/. After the session, the files persist as notebook output. Attach that output to a new notebook as a dataset to continue training.

Why does Kohya show double the epochs I set? The exact mechanism is undocumented. The UI may log optimizer steps rather than epochs, or count both text-encoder and UNet passes. Check the training log for actual step count.

Can I train IA³ or LoKR types on the Linaqruf Colab? Not through the network_category dropdown. These types require manual network_module and network_args configuration. The underlying sd-scripts may support them, but the Colab notebook does not expose them.

Is SDXL 0.9 still available on Hugging Face? SDXL 0.9 (stabilityai/stable-diffusion-xl-base-0.9) is deprecated. Current base models are SDXL 1.0 (stabilityai/stable-diffusion-xl-base-1.0) and community finetunes like AnimagineXL.

What’s the difference between --cache_latents and --cache_latents_to_disk? --cache_latents keeps all latents in RAM (fastest, uses VRAM). --cache_latents_to_disk writes them to SSD, freeing RAM. Use both together on 16 GB T4 GPUs to avoid OOM.

How much disk space do I need for Kaggle training? /kaggle/working gives 20 GB auto-saved. Use /kaggle/temp for up to 75 GB of temporary space. Download large models to /kaggle/temp/models/, but save final LoRAs to /kaggle/working/.

Sources