Inpainting & Outpainting: Masks, Denoise, Model Choice

TL;DR

• Dedicated inpainting models (SD1.5 inpainting, Fooocus's 1.28 GB patch) add 5 extra UNet input channels for masked-image + mask conditioning. Base checkpoints can inpaint but lose coherence on complex replacements (new people, new bodies).
• Masked content chooses the initial latent fill: fill/latent nothing + denoise > 0.8 = replace; original/latent noise + denoise 0.4–0.6 = modify. Match the mode to the task.
• Inpaint area: Only masked crops the masked region, upscales it to full resolution (usually 1024²), processes it, then pastes back — giving far more detail than Whole picture on large canvases.
• Mask blur 4–8 prevents hard seams. Fooocus automates this with a morphological open (dilate → erode) on every mask.
• Fooocus enables 4K+ outpainting on 12 GB VRAM by cropping to the expansion zone, processing at ≤ 1024 px, and using a lightweight inpaint head injected at the first UNet block — not a full inpainting checkpoint.
• Outpainting needs 50–100 steps, ancestral samplers (Euler a / DPM2 a), max denoise (1.0), and max CFG. Low steps or wrong sampler produce incoherent extensions.

The Model Myth: You Don't Need a Special Checkpoint for Most Inpainting

Community guides often claim "you need an inpainting model for good results." That's only half true.

SD1.5 has an official inpainting checkpoint trained for 440k additional steps with 5 extra UNet channels (4 for the VAE-encoded masked image, 1 for the binary mask). Those channels are zero-initialized after restoring the base weights, so the model learns only how to blend masked regions (SD1.5 inpainting model card).

SDXL is different. Stability AI never released a standalone SDXL inpainting checkpoint. The SDXL pipeline uses base + refiner. Community "SDXL inpainting" models are fine-tunes — some add the 5-channel architecture, others just train on masked data with the base UNet. Most behave similarly; only specific fine-tunes (Pony variants, NSFW merges) show measurable differences (Civitai article 3990).

Fooocus takes a third approach. It downloads a 1.28 GB inpaint_v26.fooocus.patch — a tiny conv head (320 channels, 5 inputs: 4 latent + 1 mask) that injects mask conditioning at the first input block of a standard SDXL UNet (Fooocus repo). Any SDXL base model works; no separate checkpoint required.

Practical rule: Use a dedicated inpainting model (SD1.5 inpainting, Fooocus) when you're replacing content entirely — new person, new object, new background. For modifying existing content (fixing faces, changing shirt color, removing a logo), a base checkpoint with the right settings works fine.

Masked Content: The Initial Fill Decides the Trajectory

The Masked content dropdown in Automatic1111 (and equivalents in ComfyUI, Fooocus) sets what the latent looks like before diffusion starts. It is not a style setting — it's the starting noise distribution.

Why denoise must match: High denoise (> 0.8) overwrites the initial latent almost completely. If you feed original (the existing face) but denoise at 0.9, you destroy the identity cues — the result looks like a different person. Low denoise (0.4–0.6) preserves the original structure while letting the prompt steer details (ThinkDiffusion aDetailer).

Inpaint Area: Only Masked vs Whole Picture

This setting controls whether the model sees the full image or just the masked crop.

• Whole picture — The entire image is resized to the target resolution (e.g., 1024×1024), the masked region is processed in context, then the result is pasted back. Global coherence is preserved, but the masked region gets fewer effective pixels.
• Only masked — The bounding box of the mask is cropped, upscaled to the target resolution, processed at full resolution, then pasted back. The edited region receives ~4× more pixel budget on a 2048 px image.

When to use which:

Source: A1111 wiki: "With Inpaint area: Only masked enabled, only the masked region is resized, and after processing it is pasted back to the original picture. This allows you to work with large pictures and render the inpainted object at a much larger resolution." (A1111 Features)

Why Edges Show Seams — And How to Fix Them

Three failure modes create visible boundaries at the mask edge:

1. Mask blur too low (0–3). The crop/paste creates a hard pixel transition. A1111 docs recommend 4–8 (A1111 Features). Fooocus applies morphological_open (dilate → erode) automatically to every mask before processing (Fooocus inpaint_worker.py).
2. Denoise mismatch. High denoise with original masked content destroys the edge pixels that should blend. Use 0.4–0.6 for modifications (ThinkDiffusion aDetailer).
3. Resolution mismatch. Whole picture on a 4K image downscales everything; the masked region loses high-frequency detail needed for a clean blend.

Fix checklist:

• Set mask blur ≥ 4 (A1111) or rely on Fooocus's auto-processing
• Match denoise to masked content: 0.4–0.6 for original, 0.8+ for fill/latent nothing
• Use Only masked for edits on images > 1024 px

Fooocus Outpainting: How It Fits 4K on 12 GB

Fooocus's outpainting avoids the VRAM explosion of full-canvas processing:

1. Crop to expansion zone. The InpaintWorker computes a bounding box (solve_abcd) around the mask + padding (Fooocus inpaint_worker.py).
2. Upscale only if needed. Crops < 1024 px are upscaled to 1024 px max; larger crops stay native.
3. Multi-scale box-blur fill. fooocus_fill blends the border with progressively larger box blurs (5, 21, 41, 81 px) for smooth latent transition (Fooocus inpaint_worker.py).
4. Morphological open on mask. cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel) softens hard edges before diffusion (Fooocus inpaint_worker.py).
5. Lightweight inpaint head. A 320-channel conv (5 in, 3×3 kernel) injects mask+latent at the first UNet input block via input_block_patch — replacing the 5-channel UNet concat used by SD1.5 inpainting models (Fooocus inpaint_worker.py).
6. Process and paste back. The diffused crop is pasted into the original canvas at full resolution.

Only the expansion zone ever occupies VRAM at 1024² latent resolution. A 4000 px outpaint on 12 GB is routine (Fooocus repo).

Outpainting Settings That Actually Work

A1111's "Poor man's outpainting" and community testing converge on one recipe (A1111 Features):

Low steps (< 30) or deterministic samplers (Euler, DPM++ 2M) produce rigid, repetitive, or incoherent extensions.

Quick Reference: Settings by Task

*DDPM (ancestral) recommended for Pony-family SDXL fine-tunes; 20–30 steps aligns with their training sampler (Civitai article 3990).

Common Errors and Fixes

FAQ

What's the difference between fill, original, latent noise, and latent nothing in inpainting? They set the initial latent before diffusion. fill averages surrounding colors; original keeps the existing pixels; latent noise adds random noise; latent nothing uses zeros. Match to task: replace → fill/latent nothing + high denoise; modify → original/latent noise + low denoise (A1111 Features).

When should I use "Inpaint area: Only masked" vs "Whole picture"? Only masked when the edit region needs maximum detail (faces, small objects, large source images). Whole picture when global lighting/color coherence matters more than local detail (background extensions, style transfer) (A1111 Features).

Do I need a dedicated inpainting model for SDXL? No. Fooocus works with any SDXL base via its 1.28 GB inpaint patch (Fooocus repo). Community SDXL inpainting models exist but most are fine-tunes; only specific variants (Pony, NSFW merges) differ measurably (Civitai article 3990). For SD1.5, the official inpainting checkpoint (5 extra UNet channels) is worth using (SD1.5 inpainting model card).

Why does my inpainted region have a visible seam? Usually mask blur too low (0–3), denoise mismatch (high denoise with original content), or Whole picture on a large image forcing downscaling. Raise mask blur to 4–8, match denoise to content mode, use Only masked for large images (A1111 Features; Fooocus inpaint_worker.py).

What denoising strength should I use for face inpainting? 0.4–0.6 with original masked content and Only masked area. Higher values destroy identity; lower values don't change enough. ThinkDiffusion's aDetailer tutorial uses ≈ 0.6 as default (ThinkDiffusion aDetailer).

Can I outpaint 4K images on 12 GB VRAM? Yes, with Fooocus. It crops the expansion zone, processes at ≤ 1024 px, uses a lightweight inpaint head (not a full inpainting UNet), and pastes back. A1111/ComfyUI full-canvas outpainting at 4K needs 24 GB+ (Fooocus repo; Fooocus inpaint_worker.py).

What sampler works best for Pony models? DDPM (ancestral) at 20–30 steps. Pony fine-tunes were trained with DDPM; ancestral samplers match their noise schedule. Not universal — check the specific model card (Civitai article 3990).

Summary

Inpainting and outpainting quality comes from matching three knobs — model, masked content + denoise, inpaint area — to the task type (replace vs modify). Dedicated inpainting models add mask conditioning via extra UNet channels (SD1.5) or a lightweight injected head (Fooocus). Base checkpoints work for modifications when settings are correct. Fooocus's crop-process-paste pipeline makes 4K outpainting viable on consumer GPUs. The most common failure — visible seams — is almost always a mask blur or denoise mismatch, not a model limitation.

Sources

Related Articles

• Stable Diffusion Sampler Comparison — How Euler a, DPM++ 2M, DDPM, and others affect inpainting quality
• VRAM Optimization for Large Resolution Generation — Techniques for 4K+ generation on 12–16 GB cards
• ControlNet for Precise Inpainting — Using depth, canny, and segmentation masks for controlled edits