Captioning Tools 2026: JoyCaption vs WD Tagger vs Manual

TL;DR

• JoyCaption Beta One gives you 12 caption styles (Descriptive, Straightforward, SD Prompt, Danbooru tags, etc.) and word-count control from 20–260 words, running in ~17 GB VRAM (bfloat16) with 8-bit/4-bit quantization options via ComfyUI.
• WD Tagger v3 (ViT or SwinV2) uses ONNX with an unfixed batch dimension, so you can process thousands of images in one call — ideal for dataset prep pipelines.
• Manual tagging remains the quality ceiling: the Civitai "Datasets for Dummies" workflow merges booru tags with AI tags, then requires a full manual pass per file.

Why captioning tool choice matters

Training diffusion models lives or dies by caption quality. A model trained on vague, repetitive, or censored captions will hallucinate, ignore prompts, or refuse NSFW concepts. The three mainstream paths — VLM captioning (JoyCaption), tagger ensembles (WD Tagger), and human-in-the-loop — each hit different points on the speed/quality/cost triangle.

What is JoyCaption Beta One?

JoyCaption Beta One is a Visual Language Model (VLM) built on LLaVA architecture: a SigLIP2-SO400M vision encoder fused to Llama-3.1-8B-Instruct. The model weights are Apache-2.0 licensed, free to download, and run locally. Unlike commercial VLMs, it was trained with explicit SFW/NSFW balance and no "cylindrical object" euphemisms.

Key specs:

Performance metrics:

JoyCaption caption styles and length control

The ComfyUI node exposes 12 caption types and a dual-mode length selector.

Caption type menu

Length control

The selector accepts descriptive labels (very short … very long) or exact word counts (20, 30, … 260 in steps of 10). This gives you deterministic template filling for both casual and technical use cases.

WD Tagger v3: architectures and batch inference

The "Datasets for Dummies" guide (Civitai 2217) describes a hybrid pipeline: scrape boorus, AI-tag everything with WD Tagger (via Kohya-ss wd14_tagger.py or Holostrawberry's Dataset Helper Colab), merge tag sets, then run a full manual QA pass per file. Teams parallelize across annotators with a shared style guide. Time estimate: ~30–60 s/image. The "Datasets for Dummies" guide describes a hybrid pipeline:

1. Scrape from boorus — pre-tagged, but noisy.
2. AI-tag everything — run WD Tagger (via Kohya-ss wd14_tagger script or Holostrawberry's Dataset Helper Colab) even on booru images.
3. Merge tag sets — PowerShell/Notepad++ concatenation, deduplicate, alphabetize.
4. Blacklist count/relation tags — 1girl, 2girls, multiple girls, brother and sister, etc. (tagger can't count relationships).
5. Manual QA pass — every file, confirm/remove/add tags. "Tag everything: every clothing item, eye color, hair color."

Time estimate: ~30–60 s/image for thorough review, but produces the cleanest training sets. Teams parallelize the manual pass across annotators.

Batch tooling comparison

Note: JoyCaption vLLM (vllm serve fancyfeast/llama-joycaption-beta-one-hf-llava --max-model-len 4096 --enable-prefix-caching) is the only production-grade path for natural language captioning.

Common errors and fixes

FAQ

What caption style for SDXL fine-tuning? Straightforward or Stable Diffusion Prompt mode. Straightforward gives clean natural language (~50–100 words) that SDXL understands well. SD Prompt mode mimics user prompts but has a 3% glitch rate and lower variety after DPO.

Can JoyCaption run on 12 GB VRAM? Yes, with 4-bit NF4 quantization (ComfyUI "Maximum Savings" preset). Expect ~1–2 img/s. For batch workloads, use vLLM on a 24 GB+ card.

Which WD Tagger backbone is better: ViT or SwinV2? SwinV2 (wd-swinv2-tagger-v3) scores higher F1 (0.4541 vs 0.4402) and has 7× more downloads. For anime/booru, prefer SwinV2; for photorealistic, ViT may generalize better.

How do I merge booru tags with AI tags without duplicates? The Civitai guide uses PowerShell to concatenate same-named .txt files from two folders, then Notepad++ regex replace \r\n → , , then deduplicate. Kohya-ss's wd14_tagger has --undesired_tags to filter specific tags; deduplication is done post-merge with text tools.

Does JoyCaption support tag augmentation? Yes, Beta One adds tag augmentation: pass comma-separated booru tags and the model integrates them (judged 6.51/10 by a SOTA VLM). Use extra_options: "tag_augmentation: tag1, tag2, tag3" in the ComfyUI node.

What's the glitch rate for JoyCaption Beta One? Overall 1.5% after DPO (SD Prompt: ~3%). Commercial VLMs (GPT-4o) report ~0.01%.

Can I use WD Tagger ONNX on CPU? Yes. onnxruntime CPU provider works; throughput ~5–10 img/s, no GPU required. Install onnxruntime≥1.17.0 and load the .onnx file.

How do I evaluate caption quality automatically? No standard automated metric exists. The JoyCaption author uses a brutal hand-eval: +1 per accurate detail, −1 per inaccurate detail, normalized against a human-written "perfect" caption. Beta One scores 67% vs Alpha Two 55% vs GPT-4o 55%.

Sources

1. JoyCaption GitHub — architecture, VRAM, licensing
2. JoyCaption ComfyUI nodes — caption types, length choices
3. JoyCaption HF model card — specs, usage snippet
4. WD ViT Tagger v3 model card — training data, validation, ONNX
5. WD SwinV2 Tagger v3 model card — architecture, metrics
6. WD v1-4 model card — legacy reference
7. Kohya-ss sd-scripts README — wd14_tagger link
8. Civitai: JoyCaption Beta One — training samples, DPO, benchmarks
9. Civitai: Datasets for Dummies — hybrid tagging workflow
10. Civitai: Prompts Creator — paid manual tagging tool

Internal links

• Dataset preparation guide
• SDXL fine-tuning parameters
• ComfyUI workflow optimization
• VLM comparison for training
• Booru scraping automation