FLUX.2 Klein Face Swap: Workflow, Color Matching, Failures

TL;DR

• LanPaint's "think mode" runs 2 inner Langevin iterations per denoising step, aligning the new face to surrounding pixels before final output
• Lab color transfer matches skin tone and lighting independently from facial structure, eliminating the pasted-in look
• The distilled 9B model needs low guidance (1.0–2.0) and only 2 think steps — more megapixels or steps produce plastic faces

How the FLUX.2 Klein Reface Pipeline Works

FLUX.2 Klein is a 9-billion-parameter rectified flow transformer that unifies text-to-image and image-to-image editing in one model, running in under 0.5 seconds on an RTX 4090 (BFL blog). The 9B variant requires ~29 GB VRAM (model card). The community face-swap workflow builds on this by layering three techniques: LanPaint inpainting with think mode, Lab* color-space transfer, and targeted LoRAs.

LanPaint Think Mode: Inner Iterations Before Denoising

Standard inpainting denoises once per step. LanPaint inserts a Langevin dynamics loop — the "think mode" — that runs multiple inner iterations before each outer denoising step. For FLUX.2 Klein, the official example uses 2 inner steps (reduced from the default 5) because distilled models become unstable with more (LanPaint repo).

Each inner iteration computes two score terms:

1. A regularization score pulling the latent toward the noisy input
2. A content-alignment score pulling masked pixels toward the known (unmasked) region and the model's own clean-image prediction

The balance is controlled by LanPaint_Lambda (default 16.0). Higher lambda binds the inpainted face more strictly to surrounding pixels, reducing color mismatches and boundary artifacts. The inner loop uses a StochasticHarmonicOscillator with friction (default 15.0) and can early-stop when the masked region's predicted clean image stabilizes (LanPaint nodes.py).

For face swap, this means the model "thinks through" how the new face should blend with existing skin tone, lighting direction, and texture before committing to the final denoised result.

Lab Color Transfer: Separating Structure from Tone

After inpainting, the swapped face often carries the source image's color cast. The workflow applies a Lab* (CIELAB) statistical transfer based on the Reinhard algorithm (color_transfer_by_lucas):

1. Convert both the inpainted face region and the target reference to Lab* space
2. Match mean and standard deviation per channel (L=luminance, a=green-red, b=blue-yellow)
3. Scale by (ref_std / content_std) and shift by (ref_mean - content_mean)
4. Blend at adjustable strength (0.0–1.0), then convert back to RGB

Because L carries brightness and structure while a/b* carry color opponency, the transfer adjusts skin tone and lighting color without blurring facial edges or texture. This is why the result stops reading as a cutout — the geometry stays from LanPaint, the chromatic context comes from the target.

Recommended Sampler: Euler (Not Euler Ancestral)

The LanPaint node lists samplers as ["euler", "euler_ancestral", "heun", ...] with euler first (LanPaint nodes.py). The original article claims "euler_ancestral / normal looks a bit better," but no FLUX-specific benchmark supports this. Euler ancestral adds stochastic noise at each step (non-convergent random walk), which can introduce fine-grained variation that fights LanPaint's content-alignment objective. Deterministic Euler (or DPM++ 2M Karras) gives reproducible face structure — important when you need the same identity across generations.

LoRA Stack for Texture and Consistency

Consistency LoRA specifically targets the drift that accumulates when you re-encode/decode or iterate — backgrounds stay aligned, reflections hold, repeated edits don't compound deformation.

Common Errors and Fixes

Where the Workflow Fails

Distilled Model Limitations

FLUX.2 Klein 9B is step-distilled to 4 inference steps. LanPaint's README explicitly warns: "degraded performance on distillation models, such as Flux.dev... use low flux guidance (1.0-2.0) to mitigate." The inner Langevin loop assumes a noise schedule that distillation compresses, so the content-alignment signal weakens. Base (undistilled) 9B/4B variants avoid this but need ~29GB VRAM and more steps.

Small Faces at Default Resolution

At default resolution, a distant face may fall below the effective detail threshold of the face-swap model. Increasing megapixels helps but produces smoother, less detailed output — the "plastic" look. The practical fix is a two-pass approach: inpainting at higher resolution with 4–8 think steps, then a FaceDetailer refinement pass.

Color Transfer Edge Cases

Lab transfer applies the target's color statistics globally. If the target has strong color casts (neon, golden hour, mixed tungsten/daylight), matching statistics bakes those casts into the face. Manual masking of the transfer region or a second correction pass (selective color grading) may be needed.

Unverified Components

The BFS Best Face Swap LoRA (civitai.com/models/2027766), Dramatic Dark Lighting LoRA, and Flux2 Anime VAE were cited in the source article but could not be independently accessed during research. Their claimed effects (identity fidelity, texture enhancement, sharpening) remain single-source.

FAQ

How does LanPaint think mode improve face swap quality? It runs inner Langevin iterations that align the masked face region to surrounding pixels before each denoising step. The face gets multiple chances to match skin tone and lighting instead of a single pass.

Why do FLUX Klein face swaps look plastic at high megapixels? The distilled model's receptive field at higher resolution produces oversmoothed outputs. Stay at 1 megapixel default; if the face is tiny, use 2 megapixels with 4–8 think steps.

What sampler should I use — euler or euler_ancestral? Use euler (deterministic). It's the first/default in LanPaint's sampler list. Euler ancestral adds random noise per step, which can destabilize face identity.

How does Lab color transfer make face swaps match lighting? Lab* separates luminance from chrominance. The transfer matches mean and standard deviation per channel independently, so skin tone adapts while facial structure stays intact.

What's the difference between FLUX.2 Klein base and distilled for inpainting? Base variants (9B/4B) are undistilled — more stable with LanPaint, need ~29GB VRAM. Distilled 9B runs in 4 steps but LanPaint degrades on it; requires lower guidance (1.0–2.0) and fewer think steps (2).

Can I use this for non-human faces? The BFS LoRA is trained on human faces. For animals or characters, IPAdapter-based transfer with a reference image works better than ONNX swap models.

Why does my mask boundary glow? LanPaint ≤ v1.4.x had a bug. Update to v1.5.0+.

Do I need the Consistency LoRA? For a single swap, probably not. For iterating (re-encoding, multiple edits, chaining passes), use it at 0.5–0.7 to prevent compounding drift.

Sources

1. BFL blog — FLUX.2 Klein announcement — model architecture, VRAM, licensing, quantization, inference speed
2. LanPaint GitHub repo — think mode mechanism, FLUX2 Klein example (2 steps), sampler defaults, distillation warning, v1.5.0 bug fix
3. LanPaint nodes.py source — KSAMPLER_NAMES list (euler first), LanPaint_KSamplerAdvanced parameters
4. color_transfer_by_lucas README — Lab* Reinhard algorithm, mean/std matching, strength blending
5. FLUX.2 Klein 9B model card — ~29GB VRAM, RTX 4090+ recommendation
6. Consistency LoRA model card — base model, purpose, strength ranges
7. HuggingFace search — Consistency LoRAs — existence verification
8. ComfyUI FLUX examples — standard sampler/scheduler/CFG/FluxGuidance defaults
9. Civitai article 27730 — source workflow claims (4 vs 8 steps, megapixels, LoRA stack, VAE)