Fixed-Seed Checkpoint and LoRA Comparison
A practical methodology for fair Stable Diffusion model comparisons — control variables and read grids honestly.
On this page
TL;DR
- The izolight testing card enforces fixed parameters across all evaluations — seed, sampler, steps, CFG, resolution, and VAE must be identical for every checkpoint/LoRA test. For background on LoRA trigger-word extraction, the ComfyUI node is the authoritative source.
- v3 uses LoRA Auto Trigger — canonical trigger tags are pulled automatically from Civitai API or model metadata.
- v2 switches to average conditionings — for Portrait and Close-up categories, the ComfyUI
ConditioningAveragenode blends text embeddings instead of concatenating prompts. - 30 LoRAs tested — covering Lava Style, Liquid Flow, Chrome Style, Bubblegum, and other style categories. See also our ComfyUI workflow guide.
Origin: From Reddit Idea to ComfyUI Workflow
The izolight article opens with a direct reference: "A few days ago I stumbled upon a reddit post with a model testing card" linking to r/StableDiffusion [[1]](https://civitai.com/articles/4474). That post (r/StableDiffusion/comments/1b39b07) is the seed for the entire methodology.
The author then built a ComfyUI implementation with two explicit improvements over the Reddit concept:
- Centrally controlled parameters — "I decided to improve this a bit with centrally controlled parameters for the generation" [[1]](https://civitai.com/articles/4474)
- LoRA and Lightning LoRA support — "...and support for LoRA and Lightning LoRA" [[1]](https://civitai.com/articles/4474)
The resulting workflow puts "all the input parameters on the left side with the final output (as a preview that can be saved) there too" [[1]](https://civitai.com/articles/4474). This layout keeps the control surface visible while the comparison grid renders.
Methodological Core: One Parameter Set, Many Models
The central principle: every variable that could affect the output must be locked identically across all test cells. For a deeper treatment, see SDXL checkpoint evaluation.
What the Card Actually Tested
The workflow evaluated three SDXL checkpoints, all using the same generation parameters: Checkpoint | Base | Version | Key Parameters | Source
|---|---|---|---|---
Realities Edge XL | SDXL Lightning | V7 (BakedVAE) | CFG 7, steps 25, DPM++ 2M SDE, seed 1 | https://civitai.com/images/7804518 Juggernaut XL | SDXL | V9 + RunDiffusionPhoto 2 | CFG 7, steps 25, DPM++ 2M SDE, seed 1 | https://civitai.com/images/7804589 Albedo Base XL | SDXL | V2.1 | CFG 7, steps 25, DPM++ 2M SDE, seed 1 | https://civitai.com/images/7805074 Generation metadata from each model confirms identical parameter values: cfgScale:7, steps:25, sampler:DPM++ 2M SDE, seed:1 [[2]](https://civitai.com/images/7804518) [[3]](https://civitai.com/images/7804589) [[4]](https://civitai.com/images/7805074). All three use SDXL-base architecture — confirmed by their consistent parameters across every published sample.
LoRA Test Suite
The 30 LoRAs span style categories from Lava Style through Chrome Style, covering:
- Artistic styles: Lava Style, Liquid Flow Style, Chocolate Wet Style, Midas Magic Grimoire, Tempest Magic Grimoire, Hedgerow Style, Wooden Blocks Style, Cardboard, Dataviz, Dippers Style
- Technical/Effect styles: Clockwork/Steampunk, Wisp Elementals, Gaelic/Celtic Patterns, Eldritch Palette Knife Painting, GlowNeon, KaleidoShatter, Lelo, Line Art, Mechanarion
- Advanced/Composite styles: Neo Futuristic Tech, Animal Exoskeleton Style, Ember Glut Style, Bling Style, Extinction/fading, Bubblegum Style, Carbon Fiber Style, Chrome Style
Each was tested with automatic trigger-word injection via the LoRA Auto Trigger node [[1]](https://civitai.com/articles/4474).
Technical Detail: LoRA Auto Trigger Architecture
The LoRA Auto Trigger project is a fork of Extraltodeus/LoadLoraWithTags and adds support for stacked LoRAs from jags111/efficiency-nodes-comfyui [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words). It extracts trigger words from two sources:
- Civitai API — for models hosted on Civitai
- Model training metadata — embedded in
.safetensorsfiles when available
This dual-source approach ensures coverage even for LoRAs distributed outside Civitai. The node outputs both civitai_tags_list and meta_tags_list as Python lists, which can be filtered and weighted via the TagsSelector node [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words). This provides flexibility in selecting trigger words for each LoRA type.
Version Evolution: Three Iterations
v1 — Baseline with Centrally Controlled Parameters
The initial release established the core architecture: a single ComfyUI workflow where all generation parameters are defined once and propagated to every cell. This eliminated the most common source of comparison artifacts — parameter drift between cells.
v2 — From Concatenation to Average Conditionings
Original prompt handling concatenated text strings: "portrait of a woman" + "soft window light". This created a single token sequence where attention could dilute across unrelated concepts.
v2 introduced average conditionings for Portrait and Close-up categories [[1]](https://civitai.com/articles/4474). The ComfyUI ConditioningAverage node computes separate text embeddings for each prompt component, then blends them into a single vector before feeding the U-Net. This preserves semantic direction from each prompt while preventing token-length penalties.
Note: The article states the change but does not explain the mechanism. The above describes standard ComfyUI ConditioningAverage behavior.
v3 — Automated Trigger-Word Injection
v3 implemented LoRA Auto Trigger [[1]](https://civitai.com/articles/4474) [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words). Instead of manual trigger words, the node extracts canonical tags from two sources: Civitai API and .safetensors training metadata.
Why this matters: When comparing LoRAs side-by-side, manual trigger-word inconsistency introduces bias. One LoRA gets its exact canonical trigger; another gets a misspelled or omitted trigger. Auto-trigger standardizes what activates each LoRA, isolating the model itself as the only variable.
Common Errors and Fixes
Error | Cause | Fix
|---|---|---
Cells produce wildly different compositions | Seed varies between cells | Lock a single seed across all cells; the izolight card uses seed 1 [[2]](https://civitai.com/images/7804518) One LoRA shows no visible effect | Trigger word missing or misspelled | Use LoRA Auto Trigger (v3) to inject canonical tags from Civitai API or metadata [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words) Portrait prompts dilute subject detail | Prompt concatenation creates token-length penalties | Switch to ConditioningAverage for prompt blending (v2 behavior) [[1]](https://civitai.com/articles/4474) Results vary between test runs | Sampler, steps, or CFG differ per model | Define parameters centrally once; propagate to every cell VAE mismatch alters colors | Different VAE per checkpoint | Use each checkpoint's baked VAE; keep all other parameters identical
FAQ
How do you fairly compare two Stable Diffusion checkpoints?
Lock every variable that could affect output: seed, sampler, steps, CFG, resolution, and VAE. Use the same prompt set across both models. The izolight testing card enforces this via centrally controlled parameters [[1]](https://civitai.com/articles/4474).
What parameters should be held constant when comparing LoRAs?
Seed, sampler, steps, CFG, resolution, VAE, and base checkpoint. Trigger words must also be standardized via LoRA Auto Trigger (v3) to avoid manual inconsistency bias [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words).
Why use a fixed seed for model comparison grids?
A fixed seed eliminates stochastic variance between cells, so visual differences can be attributed to the model rather than random noise. The izolight card uses seed 1 across all published samples [[2]](https://civitai.com/images/7804518).
What is average conditioning vs concatenation in ComfyUI?
Concatenation joins prompt strings into one token sequence; average conditioning (the ConditioningAverage node) computes separate embeddings for each prompt component, then blends them into a single vector. v2 of the card uses averaging for Portrait and Close-up categories [[1]](https://civitai.com/articles/4474).
How does LoRA Auto Trigger work?
The node extracts canonical trigger words from the Civitai API (for Civitai-hosted LoRAs) and from .safetensors training metadata. It outputs civitai_tags_list and meta_tags_list as filterable Python lists. A fork of Extraltodeus/LoadLoraWithTags with stacked-LoRA support from jags111/efficiency-nodes-comfyui [[5]](https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words).
What the Source Doesn't Reveal
Even with a detailed testing card, methodological gaps remain:
- Prompt content — prompts are embedded in the SVG/JSON attachment (not publicly accessible). The article notes that "better prompts for photo/anime comparison" could improve variety [[1]](https://civitai.com/articles/4474).
- Grid layout — the attachment encodes the cell mapping, but the article doesn't describe row/column counts or checkpoint/LoRA assignment.
- Seed count — only seed 1 appears in published metadata. Additional seeds may or may not have been tested per cell.
- Lightning LoRA details — the article mentions support but provides no explanation of how Lightning LoRA evaluation differs from standard LoRA.
- Model selection rationale — all three are SDXL photorealistic models; the selection criteria are unspecified.
- Original Reddit post — the r/StableDiffusion post returned only a verification SVG when accessed; the post content could not be read.
Common Evaluation Pitfalls vs. Card Design
These pitfalls are widely recognized in the Stable Diffusion community. The card addresses several through its design: Pitfall | The Card's Fix
|---|---
Different seeds per cell | Single fixed seed propagated to all cells Different samplers/steps/CFG per model | Centrally controlled parameters Manual trigger-word inconsistency | LoRA Auto Trigger (v3) Single-prompt verdict | Multiple test categories (Portrait, Close-up, etc.) VAE mismatch | Each checkpoint uses its baked VAE; parameters otherwise identical
Sources
- Civitai article "Model testing card" by izolight — describes the testing card workflow, version history, and lists models/LoRAs tested. https://civitai.com/articles/4474
- Realities Edge XL generation metadata (image 7804518) — shows cfgScale:7, steps:25, sampler:DPM++ 2M SDE, seed:1. https://civitai.com/images/7804518
- Juggernaut XL v9 generation metadata (image 7804589) — identical parameter set. https://civitai.com/images/7804589
- Albedo Base XL v2.1 generation metadata (image 7805074) — identical parameter set. https://civitai.com/images/7805074
- ComfyUI-Lora-Auto-Trigger-Words GitHub repository — documents trigger-word extraction from Civitai API and model metadata. https://github.com/idrirap/ComfyUI-Lora-Auto-Trigger-Words