Pony Diffusion XL: Native Tags, Testing & LoRA

How Pony Diffusion XL Learns Characters

Pony Diffusion V6 XL is an SDXL finetune trained on roughly 2.6 million images with a 1:1 split across anime, cartoon, furry, and pony datasets, and a 1:1 split across safe, questionable, and explicit ratings. About 50 % of those images carry detailed natural-language captions; the rest carry structured Danbooru-style tags. Artist names were stripped during preprocessing.

This dual-format training creates two parallel pathways for character knowledge:

The model card explicitly states it "understands both" prompting styles. In practice, the tag pathway is more precise for obscure characters; the natural-language pathway handles outfit variations and poses more flexibly.

The score-tag quality gate

Every V6 XL generation should begin with the full quality string:

score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, <your prompt>, tag1, tag2

The model card notes that earlier Pony versions used only score_9; the longer string is a V6 XL training artifact [^1]. High-score images in the training set disproportionately feature well-tagged popular characters, so the full string activates the subspace where character fidelity is strongest.

Source-style tags as domain selectors

Four special tags switch the model between visual domains:

Character recognition varies by domain. A character well-represented in anime training data (e.g., Zelda) will render more faithfully with source_anime than with source_cartoon.

CLIP skip 2 is non-negotiable

The model card warns: "Make sure you load this model with clip skip 2 (or -2 in some software), otherwise you will be getting low quality blobs" [^1]. CLIP skip 2 uses the penultimate transformer layer instead of the final one, preserving token-level detail (character names, specific descriptors) that the final layer would abstract away.

Which Characters Are Baked In (Verified Tiers)

The community-compiled list at Civitai article #5102 color-codes characters by reliability. The underlying data comes from the ertyased2/characters_list GitHub repository, which contains:

• 8,203 unique training tags with frequency counts
• 200+ tested character entries with series and trait tags
• 1,497 formatted anime character tags in character_name_(series), series, traits format

Green tier — reliable without extra prompting

These characters have thousands of tagged training images and render correctly with just the character name + series tag.

Yellow tier — works with series tag or extra prompting

The root cause is tag ambiguity — "Mario" appears in artist names, game titles, and character tags — and visual simplicity (red cap + overalls ≈ generic tags).

Red tier — LoRA required

Rule of thumb from the compiler: "Characters in red are not recognized and are listed for those looking to make LoRAs of these characters."

How to Test Whether a Character Is Baked In

Automated method (used by the list compiler)

The ertyased2 repository includes main.py, a Selenium script that drives Automatic1111's web UI:

1. Load Pony V6 XL with CLIP skip 2
2. Prompt template: score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, [character_name]
3. Generate 50 images at 1296 × 1032, Euler a, 20 steps, random 2-character combinations
4. Visually inspect — does the output resemble the source character?

Manual protocol (recommended for users)

1. Load Pony V6 XL + SDXL VAE, CLIP skip 2
2. Prompt: score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, [character name], [series name]
3. Generate 4–8 images at 1024 × 1024, Euler a, 25 steps, CFG 7
4. Evaluate: ≥ 50 % of generations show recognizable features (hair color, outfit, species) without LoRA

Failure modes to watch for:

• Generic anime girl (female bias)
• Wrong species (Donkey Kong → donkey)
• Missing key features
• Pseudo-signature artifacts overwhelming the image

When You Still Need a Character LoRA

Common Errors and Fixes

FAQ

Does Pony Diffusion XL know [character]?

Check the tags.txt frequency for character_name_(series). If count > 500, it likely works green-tier; 100–500 = yellow-tier (needs series tag); < 100 = red-tier (needs LoRA). The Civitai article #5102 list is the fastest lookup.

Pony Diffusion XL Mario not working — why?

Bare mario (402 count) is ambiguous. Use mario_(series) (2,650 count) + full score string + CLIP skip 2. The model knows Mario; the token is just noisy.

What do yellow tier characters mean?

"Pony kind of gets the idea but will require additional prompting to work well." Typically: add series tag, specify outfit, or use source_anime.

When do I need a character LoRA for Pony Diffusion?

Red-tier characters (Funky Kong, minor NPCs), exact likeness needs, or when native tags produce wrong species/style despite correct prompting.

How to test if Pony knows a character?

Generate 4–8 images at 1024² with full score string, CLIP skip 2, character + series name. ≥ 50 % recognizable = baked in.

Pony Diffusion native character tags format?

character_name_(series), series, 1girl/1boy, hair_color, eye_color — see anime_characters.txt (1,497 entries).

Does Pony V6 XL improve character recognition over V5?

Model card only documents V5.5 for pseudo-signature fix; no character-recognition comparison published.

Sources

[^1]: Pony Diffusion V6 XL model card — Civitai API model 257749 [^2]: Civitai article #5102 — "Pony Diffusion XL Recognized Characters List" [^3]: ertyased2/characters_list — tags.txt (8,203 training tags with frequencies) [^4]: ertyased2/characters_list — characters.txt (200+ tested character entries) [^5]: ertyased2/characters_list — anime_characters.txt (1,497 formatted tags) [^6]: ertyased2/characters_list — main.py (Selenium test methodology)

Further Reading

• How to train a character LoRA for Pony Diffusion
• Pony Diffusion V6 XL prompt template deep dive
• SDXL CLIP skip explained