Prompting Fundamentals That Transfer Across Models

TL;DR

• Tokenization is BPE with CLIP's 77-token window (75 usable) and 75-token SD WebUI chunks.
• Positional embeddings exist but the "earlier tokens stronger" rule is a heuristic, not a documented gradient.
• Emphasis syntax (() and (token:weight)) modifies the conditioning tensor after CLIP encoding.
• SDXL uses dual CLIPs for broader semantics; Flux adds T5 for longer context and different tokenization (SentencePiece).
• Lighting prompts (back lighting, rim lighting, volumetric lighting, etc.) work best with explicit light sources.

What the Text Encoder Actually Receives

When you type a prompt, SD 1.5, SDXL, and Flux first tokenize your text into integer IDs. All three use BPE, but details diverge:

SD 1.5 and SDXL's CLIP-ViT/L encoder has a fixed context length of 77 tokens (start/end included), leaving 75 tokens for actual prompt content. SD WebUI splits prompts longer than 75 tokens into 75-token chunks, each processed independently through CLIP, then concatenated before reaching the UNet. Tokens near chunk boundaries lose cross-chunk attention; the BREAK keyword forces a boundary at a specific position. The "early tokens stronger" belief is partly a side effect of chunking.

Sources: CLIP repo clip.tokenize(context_length=77); SD WebUI Features wiki.

For the model-specific vocabularies built on top of these mechanics, see our guides to Pony and Illustrious quality tags and negative prompts per model family.

Why Token Order Matters

Community guides claim: "Tokens at the beginning have more weight." The mechanism is more nuanced:

Positional Embeddings ≠ Power Gradient

CLIP adds learned positional embeddings to each token. These encode absolute position (0–76) so the transformer knows token order. However, CLIP's self-attention is full bi-directional — every token attends to every other within its chunk. There is no hardcoded decay where position 0 > position 74.

What does create position-dependent effects:

1. Chunk boundaries — tokens at positions 74–75 cannot attend to tokens in the next chunk
2. Learned attention patterns — contrastive pre-training may weight certain syntactic positions more heavily
3. Prompt construction habits — users naturally put the subject first

Status: Unverified. No official documentation confirms a simple positional power gradient.

Sources: SD WebUI Features (chunking); CLIP paper arXiv:2103.00020.

Emphasis Syntax: What () and (token:weight) Actually Do

The parentheses and numeric weight syntax are SD WebUI features, not CLIP features. They operate after the text encoder.

How It Works

1. CLIP tokenizes and encodes your prompt → produces a conditioning tensor
2. The WebUI prompt parser detects (token:weight) patterns
3. The parser multiplies the embedding vectors for those tokens by weight
4. The modified tensor goes to the UNet's cross-attention layers

Source: prompt_parser.py defines round_bracket_multiplier = 1.1 and square_bracket_multiplier = 1 / 1.1.

Practical Limits

Community testing (evakenshin, DreamShaper 6/7) reports weights above ~1.5 often distort faces. Different models tolerate different ranges.

Source: evakenshin, "我权重开到了1.5倍...容易脸崩" (Civitai article 1372).

SDXL's Dual Encoder: Why Two CLIPs?

SDXL concatenates outputs of two independently trained text encoders:

1. CLIP-ViT/L (OpenAI) — 77-token context, BPE
2. OpenCLIP-ViT/G (LAION) — 77-token context, BPE

The UNet receives a conditioning tensor of shape (batch, 77, 2×hidden_dim). The refiner model uses only OpenCLIP-ViT/G. Each encoder captures different semantic nuances, so prompt engineering that works for one may not optimize the other.

Sources: SDXL model card; Stability AI generative-models README. Once the prompt is settled, the sampler and CFG choices decide how faithfully it lands — covered in samplers and schedulers.

Flux: CLIP + T5 Changes the Game

Flux replaces dual-CLIP with CLIP-ViT/L + T5-XXL. Key differences:

The HFEmbedder class routes to CLIP or T5 based on model version. Flux prompts can be much longer than 77 tokens, but tokenization differs — you cannot copy-paste SD prompts and expect identical token alignment.

Source: Flux conditioner.py HFEmbedder class.

Key principle: Any light effect requires a light source; adding light source object prompts produces better results.

Source: Civitai article 1372 (DreamShaper 6/7, SD 1.5).

Common Errors and Fixes

FAQ

Why does moving tokens around change the image in Stable Diffusion?

Token order affects positional embeddings and, past 75 tokens, which chunk a token lands in. Within a chunk, full self-attention means every token can influence every other — but the learned attention patterns may favor certain syntactic positions. The effect is real but not a simple "earlier = stronger" rule.

Does prompt order matter in SDXL vs SD 1.5?

The chunking behavior is identical (both use CLIP-ViT/L with 75-token chunks). SDXL adds a second encoder (OpenCLIP-ViT/G) that processes the same tokens independently. Order effects exist in both, but the dual-encoder concatenation can amplify or dampen them unpredictably.

What text encoder does Flux use for prompting?

Flux uses two encoders simultaneously: CLIP-ViT/L (OpenAI, BPE, 77 tokens) and T5-XXL (Google, SentencePiece, configurable length). Both condition the model; T5 handles longer, more complex descriptions.

Why are some tokens ignored in long Stable Diffusion prompts?

Prompts exceeding 75 tokens are split into chunks. Tokens at chunk boundaries lose cross-chunk attention. Tokens in later chunks only influence the UNet through the concatenated embedding — they never "see" tokens in earlier chunks during CLIP encoding.

How does the 75-token CLIP limit affect my prompts?

Plan your first 75 tokens to contain the subject, key style modifiers, and critical lighting/composition tokens. Use BREAK to force a chunk boundary to control chunking. The SD WebUI supports prompts longer than 150 tokens (three or more chunks) without truncation.

Can I use (token:weight) syntax in Flux?

Unconfirmed. The syntax is implemented in SD WebUI's prompt_parser.py. Flux uses a different inference stack (BFL/ComfyUI). Test in your workflow; do not assume it works.

What's the difference between rim lighting and back lighting?

Both create edge separation. rim lighting emphasizes a thin bright outline; back lighting places the light source behind the subject, often creating a silhouette with edge glow. Use both at moderate weights (1.1–1.3) for stronger separation.

Sources

• github.com — cited inline above.
• github.com — cited inline above.
• github.com — cited inline above.
• huggingface.co — cited inline above.
• github.com — cited inline above.
• github.com — cited inline above.
• github.com — cited inline above.