Merge Stable Diffusion Checkpoints: Methods & Block Weights

TL;DR

• Pick the method by goal: Weighted Sum blends two models along a line; Add Difference injects one model's learned direction into another; block weights set the ratio per U-Net stage so you can take one model's faces while keeping another's lighting.
• Sum Twice uses β as the weight on the third model, not α·β. The operation is a nested blend — lerp(lerp(A,B,α),C,β) — so the third model enters with weight β. Several community guides misstate this as α·β.
• Merging runs on CPU and produces a standalone checkpoint; a LoRA is a runtime modifier that cannot run alone. Use a merge when you want one self-contained model; use a LoRA to swap effects on the fly.

Merging two or more Stable Diffusion checkpoints blends their trained weights with arithmetic — no GPU, no training data, no optimizer — and outputs a new .safetensors file. The dominant tools — AUTOMATIC1111's built-in merger, SuperMerger, the Chattiori Model Merger, and kohya-ss — share the same core math. The examples target SDXL; SD 1.5 and 2.x differ only in block count.

The four merge methods that matter

What is the difference between Weighted Sum and Add Difference?

Weighted Sum linearly interpolates between two models — at α=0 the output is model A, at α=1 it is model B. AUTOMATIC1111 defines it as A * (1 - M) + B * M; CMM's merge.py implements it with torch.lerp(A, B, alpha) — the same operation under a different flag.

Add Difference takes three models. It scales the vector from C to B by α and adds it onto A — A + (B - C) * M. Use it when a base model already has the look you want and you want to graft a concept onto it without diluting the base.

Why Sum Twice is not the same as Triple Sum

Both take three models and two ratios but differ in structure.

Triple Sum blends all three models in one pass: A keeps weight 1−α−β, B gets α, C gets β. Sum Twice blends twice — first A and B at α, then that intermediate result with C at β. The CMM source makes the nested structure explicit:

def sum_twice(theta0, theta1, theta2, alpha, beta):
    out = torch.empty_like(theta0)
    torch.lerp(theta0, theta1, float(alpha), out=out)   # out = lerp(A,B,alpha)
    torch.lerp(out,   theta2, float(beta),  out=out)    # out = lerp(out,C,beta)
    return out

Expanding that second lerp shows the third model enters with weight β, not α·β. SuperMerger's documentation agrees, publishing the formula as $(1-\beta)((1-\alpha)A+\alpha B)+\beta C$. Some community guides write C's coefficient as α·β — that would only hold if the second lerp used α·β, which it does not. When tuning a Sum Twice merge, treat β as the dial controlling how much model C shows up.

Block weights: steering the merge by U-Net stage

A flat alpha applies the same ratio to every tensor. Block-weighted merging — "MBW" in SuperMerger, a 20-value string in CMM — sets a different ratio for each U-Net block, so one block can lean toward model A while another leans toward B.

What do the IN, OUT, MID, and BASE blocks control?

The Stable Diffusion U-Net is a stack of stages, and a block-weighted merge addresses them one at a time. The official block order for SDXL is:

BASE, IN00, IN01, IN02, IN03, IN04, IN05, IN06, IN07, IN08, MID00, OUT00, OUT01, OUT02, OUT03, OUT04, OUT05, OUT06, OUT07, OUT08

That is 20 values for SDXL; SD 1.5 and 2.x grow to 26 (IN00–IN11 and OUT00–OUT11). BASE is the text encoder; IN, MID, and OUT are the U-Net's downsampling, bottleneck, and upsampling stages. SuperMerger frames the payoff plainly: "each block may correspond to background depiction, characters, art style, etc., you can create a variety of merged models by changing the ratio for each block."

At a finer grain, SuperMerger's elemental-merge notes single out the attention layers: "attn is responsible for the face and clothing information. The IN07, OUT03, OUT04, and OUT05 layers seem to have a particularly strong influence." Beyond that, precise per-block role tables (e.g. "IN00 controls fingers") come from individual practitioners and are not documented officially — treat them as starting points to test, not established fact.

Writing a block-weight vector

In CMM the alpha argument accepts a comma-separated string matching that block order. A Sum Twice example with 20 values:

CM A + B +S C "0,1,1,0.2,0.2,0.3,0.3,0,0,0,0,0.2,0.1,0.1,0.1,0.4,0.4,0.4,0.4,0.4" \
   "0,0,0,0.3,0.3,0.2,0.2,0.3,0,0,0,0.1,0.1,0.1,0.2,0.2,0.2,0.2,0.2,0.2" Result

Here BASE and the early IN blocks lean toward B, the later IN and MID blocks stay near A, and the OUT blocks pull in C — a pattern for keeping one model's anatomy while borrowing another's finish.

When to merge vs use a LoRA

When should I merge two checkpoints instead of training or using a LoRA?

A merge produces a self-contained checkpoint with no adapter file. A LoRA (Low-Rank Adaptation) freezes the base weights and injects two small low-rank matrices per adapted layer, applied on top of a checkpoint at inference. The original LoRA paper shows this cuts trainable parameters by roughly 10,000× for large models; in practice a Stable Diffusion LoRA is 2–200 MB against a 2–7 GB checkpoint — "usually 10 to 100 times smaller," per the stable-diffusion-art guide.

That size gap is the deciding factor. Reach for a merge when you want a single model that "just works" without dialing a LoRA weight every prompt — a character mix, a permanent style fusion, or a base to redistribute. Reach for a LoRA when you want to layer effects on the fly, try many styles without storing full checkpoints, or stack several modifiers on one image. Because a LoRA "cannot be used alone" and "must be used with a model checkpoint file," it is the wrong tool when your goal is a standalone deliverable.

Baking a LoRA into a checkpoint

The two approaches meet at "bake." SuperMerger's Merge-to-Checkpoint feature and CMM's lora_bake.py (the Universal Model Merge Scripter's LB operator) fold one or more LoRAs permanently into a checkpoint's weights, with ratios that "can be any value, including negative values. There is no restriction that the total must sum to 1." Baking trades runtime flexibility for a self-contained model. For LoRA training itself, see the free LoRA training guide and the Flux LoRA parameter reference.

DARE: dropping and rescaling delta parameters

What is DARE merge and why does it drop and rescale parameters?

DARE (Drop And REscale) comes from the language-model paper "Language Models are Super Mario" and was adapted to Stable Diffusion by the safetensors-merge-supermario repository (the paper's reference implementation is MergeLM). It randomly zeros a fraction p of the delta parameters (the difference between two models' weights) and rescales the survivors by $1/(1-p)$ to preserve the layer's overall magnitude.

The paper's reasoning is specific to supervised fine-tuning: SFT changes very few parameters, with small magnitudes, and most of that change is redundant. Removing 90% or even 99% of those deltas and rescaling the rest "approximately preserves the original embeddings." Two cautions follow:

• Rescaling is mandatory. "The rescale operation is crucial to guarantee the success of DARE, and dropping 30% or 40% delta parameters without rescaling would noticeably lead to worse results."
• DARE assumes small deltas. The paper reports SFT deltas typically within ~0.002, and warns that "once models undergo continuous pre-training, the delta parameters can rapidly reach around 0.03, making DARE infeasible." The image-model adaptation applies the same arithmetic to (model1 − model0), but the 90–99% pruning guarantee was measured on language tasks, not diffusion checkpoints — do not assume the same rates carry over.

In CMM the +D operator maps to the DARE mode, with alpha as the blend strength and beta as the drop probability p, reproducible via --seed.

Do I need a GPU to merge Stable Diffusion checkpoints?

No. Merging is pure weight arithmetic — add, scale, lerp — run key-by-key over the state dicts, with no forward pass, data, or optimizer. CMM "runs on CPU by default (GPU optional)," and the DARE paper frames the method as needing "no GPUs." The real constraint is memory: SuperMerger's author notes "a minimum of 64GB of CPU memory is required for the XL model merge," and running other apps alongside a big SDXL merge can destabilize the system.

Common errors and fixes

How to evaluate a merged model

There is no automated score for "is this merge good." The standard workflow is an XYZ grid: sweep the merge ratio(s) against a fixed prompt and seed so you can eyeball how each alpha, beta, or block-weight preset changes the output without saving a new checkpoint each time. SuperMerger's XYZ Plot "performs sequential merge image generation" across all merge modes.

For a predictive read, SuperMerger's Analysis tab quantifies the per-block difference between two models with cosine similarity, and its "ASimilality" mode on qkv tensors "gives a result that is closer to the difference in output images." Use it before merging to predict which blocks will move, then confirm with the XYZ grid using a fixed sampler, scheduler, and seed (see the samplers and schedulers reference) so any visible change comes from the merge, not the generation settings.

FAQ

What is the difference between Weighted Sum and Add Difference when merging checkpoints?

Weighted Sum blends two models with (1−α)A + αB; Add Difference adds a scaled direction A + α(B − C), using C as the baseline that B was fine-tuned from. Use WS to mix two similar models; use AD to graft a concept onto a base you want to preserve.

Why does Sum Twice give model C a weight of β and not α·β?

Sum Twice is a nested lerp: lerp(lerp(A,B,α),C,β), which expands to (1−β)((1−α)A + αB) + βC. C's coefficient is β, not α·β — the latter would only hold if the second lerp used α·β as its ratio, and it uses β.

What do the IN, OUT, MID, and BASE blocks control in a Stable Diffusion merge?

BASE is the text encoder; IN, MID, and OUT are the U-Net's downsampling, bottleneck, and upsampling stages. SuperMerger documents that blocks "may correspond to background depiction, characters, art style," and that attention layers plus IN07/OUT03/OUT04/OUT05 strongly affect faces and clothing. More granular per-block role tables are practitioner intuition, not official documentation.

Do I need a GPU to merge Stable Diffusion checkpoints?

No. Merging is arithmetic on the weight tensors with no forward pass or training; CMM runs on CPU by default, and the DARE paper describes the method as needing "no GPUs." The real constraint is system RAM — SuperMerger flags 64 GB as a minimum for SDXL merges.

When should I merge two checkpoints instead of using a LoRA?

Merge when you want a single self-contained checkpoint — a permanent style fusion or a model you'll redistribute. Use a LoRA when you want to swap effects per prompt or keep storage small, since a LoRA cannot run alone and is roughly 10–100× smaller than a checkpoint.

What is DARE merge and why does it drop and rescale parameters?

DARE zeros a fraction p of the delta parameters between two models and rescales the survivors by $1/(1-p)$. The paper shows SFT deltas are small and highly redundant, so most can be removed without losing ability — but only if rescaling stays on and deltas are genuinely small.

How do I evaluate whether a merged checkpoint is better than its parents?

Run an XYZ grid sweeping the merge ratios against a fixed prompt and seed, keeping sampler, steps, and CFG constant. For a predictive check, SuperMerger's Analysis tab measures per-block cosine similarity, with an ASimilality mode on qkv tensors that tracks the output-image difference.

Can I bake a LoRA permanently into a checkpoint, and how is that different from a merge?

Yes. Bake folds a LoRA's low-rank deltas into the checkpoint's weights so the result runs with no adapter file; multiple LoRAs can be baked at once with ratios that need not sum to 1. The difference from a plain merge is the input — a LoRA's small delta matrices versus two full models' weights.

Sources

• Chattiori Model Merger — merge_modes.py and Utils.py (merge formula implementations)
• SuperMerger extension — README, calcmode docs, elemental-merge notes
• AUTOMATIC1111 webui — ui_checkpoint_merger.py (Weighted Sum / Add Difference UI strings)
• kohya-ss — merge_models.py (weighted-sum merge script)
• DARE paper — "Language Models are Super Mario," arXiv:2311.03099; reference implementation MergeLM; Stable Diffusion adaptation safetensors-merge-supermario
• LoRA paper — "LoRA: Low-Rank Adaptation of Large Language Models," arXiv:2106.09685; practitioner guide stable-diffusion-art.com/lora
• Universal Model Merge Scripter (MMS) — GitHub repo (the LB bake operator)