Pony Diffusion XL: Native Tags, Testing & LoRA
Which characters Pony XL knows natively, how to test baked-in tags, and when a LoRA is still required.
On this page
How Pony Diffusion XL Learns Characters
Pony Diffusion V6 XL is an SDXL finetune trained on roughly 2.6 million images with a 1:1 split across anime, cartoon, furry, and pony datasets, and a 1:1 split across safe, questionable, and explicit ratings. About 50 % of those images carry detailed natural-language captions; the rest carry structured Danbooru-style tags. Artist names were stripped during preprocessing.
This dual-format training creates two parallel pathways for character knowledge: Pathway | What it captures | Example token
|---------|------------------|---------------
Structured tags | character_name_(series), 1girl, blonde_hair, blue_eyes | princess_peach, mario_(series) Natural language | "Princess Peach from Mario wearing a pink dress" | Free-text description The model card explicitly states it "understands both" prompting styles. In practice, the tag pathway is more precise for obscure characters; the natural-language pathway handles outfit variations and poses more flexibly.
The score-tag quality gate
Every V6 XL generation should begin with the full quality string:
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, <your prompt>, tag1, tag2The model card notes that earlier Pony versions used only score_9; the longer string is a V6 XL training artifact [^1]. High-score images in the training set disproportionately feature well-tagged popular characters, so the full string activates the subspace where character fidelity is strongest.
Source-style tags as domain selectors
Four special tags switch the model between visual domains: Tag | Domain activated
|-----|------------------
source_anime | Anime-style character knowledge source_cartoon | Western cartoon style source_furry | Anthropomorphic/furry style source_pony | My Little Pony style Character recognition varies by domain. A character well-represented in anime training data (e.g., Zelda) will render more faithfully with source_anime than with source_cartoon.
CLIP skip 2 is non-negotiable
The model card warns: "Make sure you load this model with clip skip 2 (or -2 in some software), otherwise you will be getting low quality blobs" [^1]. CLIP skip 2 uses the penultimate transformer layer instead of the final one, preserving token-level detail (character names, specific descriptors) that the final layer would abstract away.
Which Characters Are Baked In (Verified Tiers)
The community-compiled list at Civitai article #5102 color-codes characters by reliability. The underlying data comes from the ertyased2/characters_list GitHub repository, which contains:
- 8,203 unique training tags with frequency counts
- 200+ tested character entries with series and trait tags
- 1,497 formatted anime character tags in
character_name_(series), series, traitsformat
Green tier — reliable without extra prompting
Franchise | Examples (verified in tags.txt)
|-----------|---------------------------------
Super Mario | Princess Peach (princess_peach 1,441), Princess Daisy, Rosalina, Bowser, Luigi, Yoshi, Toadette Legend of Zelda | Zelda (princess_zelda 2,781), Link (link 2,484), Ganon/Ganondorf, Midna, Mipha, Urbosa Sonic | Sonic (sonic_(series) 414), Amy, Shadow, Knuckles, Tails, Rouge Pokémon (PokéGirls) | Cynthia, Misty, Dawn, May, Serena, Lillie, Marnie, Nessa, Sonia, Gloria, Penny, Nemona Pokémon (species) | Pikachu (764), Eevee (493), all Eeveelutions, Gardevoir (1,903), Charizard, Lucario These characters have thousands of tagged training images and render correctly with just the character name + series tag.
Yellow tier — works with series tag or extra prompting
Character | Issue | Fix
|-----------|-------|-----
Mario | Generates generic man in red | Add mario_(series) (2,650 count) instead of bare mario (402) Jasmine (Pokémon) | "Pretty inconsistent" | Add source_anime, specify outfit Donkey Kong | Generates actual donkey | Use donkey_kong_(character), mario_(series), gorilla The root cause is tag ambiguity — "Mario" appears in artist names, game titles, and character tags — and visual simplicity (red cap + overalls ≈ generic tags).
Red tier — LoRA required
Character | Reason
|-----------|--------
Funky Kong | < 50 tagged images; no token association formed Princess King Boo | Minor NPC, insufficient training samples Most male / non-humanoid / furry characters | Training data skewed toward anime girls
Rule of thumb from the compiler: "Characters in red are not recognized and are listed for those looking to make LoRAs of these characters."
How to Test Whether a Character Is Baked In
Automated method (used by the list compiler)
The ertyased2 repository includes main.py, a Selenium script that drives Automatic1111's web UI:
- Load Pony V6 XL with CLIP skip 2
- Prompt template:
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, [character_name] - Generate 50 images at 1296 × 1032, Euler a, 20 steps, random 2-character combinations
- Visually inspect — does the output resemble the source character?
Manual protocol (recommended for users)
- Load Pony V6 XL + SDXL VAE, CLIP skip 2
- Prompt:
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, [character name], [series name] - Generate 4–8 images at 1024 × 1024, Euler a, 25 steps, CFG 7
- Evaluate: ≥ 50 % of generations show recognizable features (hair color, outfit, species) without LoRA
Failure modes to watch for:
- Generic anime girl (female bias)
- Wrong species (Donkey Kong → donkey)
- Missing key features
- Pseudo-signature artifacts overwhelming the image
When You Still Need a Character LoRA
Situation | Why native tags fail | Example
|-----------|---------------------|---------
Character absent from training data (< ~50 tagged images) | No token→visual association formed | Funky Kong, Princess King Boo Character exists but inconsistent (yellow tier) | Insufficient samples / tag noise | Jasmine (Pokémon) Wrong species or style | Tag collision / domain mismatch | Donkey Kong → donkey; Mario → generic man Specific outfit/version not dominant | Model averages all appearances | Zelda — multiple canonical outfits Exact likeness required (cosplay, style match) | Native knowledge is conceptual, not pixel-perfect | Any 1:1 reference match need Male / non-humanoid / furry | Training data skewed toward anime girls | Most male characters are yellow/red tier
Common Errors and Fixes
Symptom | Cause | Fix | Evidence | Source
|---------|-------|-----|----------|--------
Mario → generic man in red | mario token ambiguous (artist, game, character) | Use mario_(series) (2,650) not bare mario (402) | tags.txt frequencies | https://raw.githubusercontent.com/ertyased2/characters_list/main/tags.txt Donkey Kong → actual donkey | donkey_kong conflates with animal tags; furry bias | Prompt donkey_kong_(character), mario_(series), gorilla or LoRA | tags.txt lacks standalone donkey_kong | https://raw.githubusercontent.com/ertyased2/characters_list/main/tags.txt Funky Kong → unrecognizable | < 50 training images | Train LoRA; no native token exists | Red tier in source list | https://civitai.com/articles/5102 Pseudo signatures on outputs | Artist signatures/watermarks in training data | Use V5.5 variant or inpaint removal | Model card: "training issue" | https://civitai.com/api/v1/models/257749
FAQ
Does Pony Diffusion XL know [character]?
Check the tags.txt frequency for character_name_(series). If count > 500, it likely works green-tier; 100–500 = yellow-tier (needs series tag); < 100 = red-tier (needs LoRA). The Civitai article #5102 list is the fastest lookup.
Pony Diffusion XL Mario not working — why?
Bare mario (402 count) is ambiguous. Use mario_(series) (2,650 count) + full score string + CLIP skip 2. The model knows Mario; the token is just noisy.
What do yellow tier characters mean?
"Pony kind of gets the idea but will require additional prompting to work well." Typically: add series tag, specify outfit, or use source_anime.
When do I need a character LoRA for Pony Diffusion?
Red-tier characters (Funky Kong, minor NPCs), exact likeness needs, or when native tags produce wrong species/style despite correct prompting.
How to test if Pony knows a character?
Generate 4–8 images at 1024² with full score string, CLIP skip 2, character + series name. ≥ 50 % recognizable = baked in.
Pony Diffusion native character tags format?
character_name_(series), series, 1girl/1boy, hair_color, eye_color — see anime_characters.txt (1,497 entries).
Does Pony V6 XL improve character recognition over V5?
Model card only documents V5.5 for pseudo-signature fix; no character-recognition comparison published.
Sources
[^1]: Pony Diffusion V6 XL model card — Civitai API model 257749 [^2]: Civitai article #5102 — "Pony Diffusion XL Recognized Characters List" [^3]: ertyased2/characters_list — tags.txt (8,203 training tags with frequencies) [^4]: ertyased2/characters_list — characters.txt (200+ tested character entries) [^5]: ertyased2/characters_list — anime_characters.txt (1,497 formatted tags) [^6]: ertyased2/characters_list — main.py (Selenium test methodology)