ArtificialGuyBR

Home / Blog / Prompting and control

Prompting Fundamentals That Transfer Across Models

What the text encoder receives, why token order matters, and which prompting habits survive SD, SDXL, and Flux.

7 sources cited Prompting and control

TL;DR


What the Text Encoder Actually Receives

When you type a prompt, SD 1.5, SDXL, and Flux first tokenize your text into integer IDs. All three use BPE, but details diverge: Model Family | Text Encoder(s) | Tokenization | Max Prompt

|---|---|---|---

SD 1.5 | CLIP-ViT/L | BPE | 75 tokens SDXL | CLIP-ViT/L + OpenCLIP-ViT/G | BPE (both) | 75 each, concatenated Flux | CLIP-ViT/L + T5-XXL | BPE + SentencePiece | 75 (CLIP) + configurable (T5) SD 1.5 and SDXL's CLIP-ViT/L encoder has a fixed context length of 77 tokens (start/end included), leaving 75 tokens for actual prompt content. SD WebUI splits prompts longer than 75 tokens into 75-token chunks, each processed independently through CLIP, then concatenated before reaching the UNet. Tokens near chunk boundaries lose cross-chunk attention; the BREAK keyword forces a boundary at a specific position. The "early tokens stronger" belief is partly a side effect of chunking.

Sources: CLIP repo clip.tokenize(context_length=77); SD WebUI Features wiki.

For the model-specific vocabularies built on top of these mechanics, see our guides to Pony and Illustrious quality tags and negative prompts per model family.

Why Token Order Matters

Community guides claim: "Tokens at the beginning have more weight." The mechanism is more nuanced:

Positional Embeddings ≠ Power Gradient

CLIP adds learned positional embeddings to each token. These encode absolute position (0–76) so the transformer knows token order. However, CLIP's self-attention is full bi-directional — every token attends to every other within its chunk. There is no hardcoded decay where position 0 > position 74.

What does create position-dependent effects:

  1. Chunk boundaries — tokens at positions 74–75 cannot attend to tokens in the next chunk
  2. Learned attention patterns — contrastive pre-training may weight certain syntactic positions more heavily
  3. Prompt construction habits — users naturally put the subject first

Status: Unverified. No official documentation confirms a simple positional power gradient.

Sources: SD WebUI Features (chunking); CLIP paper arXiv:2103.00020.

Emphasis Syntax: What () and (token:weight) Actually Do

The parentheses and numeric weight syntax are SD WebUI features, not CLIP features. They operate after the text encoder.

How It Works

  1. CLIP tokenizes and encodes your prompt → produces a conditioning tensor
  2. The WebUI prompt parser detects (token:weight) patterns
  3. The parser multiplies the embedding vectors for those tokens by weight
  4. The modified tensor goes to the UNet's cross-attention layers

Syntax | Effect | Weight Range

|---|---|---

(word) | 1.1× emphasis | implicit (word:1.3) | 1.3× emphasis | typical 0.5–1.5 [word] | 0.909× de-emphasis | implicit (word:0.7) | 0.7× de-emphasis | explicit Source: prompt_parser.py defines round_bracket_multiplier = 1.1 and square_bracket_multiplier = 1 / 1.1.

Practical Limits

Community testing (evakenshin, DreamShaper 6/7) reports weights above ~1.5 often distort faces. Different models tolerate different ranges.

Source: evakenshin, "我权重开到了1.5倍...容易脸崩" (Civitai article 1372).

SDXL's Dual Encoder: Why Two CLIPs?

SDXL concatenates outputs of two independently trained text encoders:

  1. CLIP-ViT/L (OpenAI) — 77-token context, BPE
  2. OpenCLIP-ViT/G (LAION) — 77-token context, BPE

The UNet receives a conditioning tensor of shape (batch, 77, 2×hidden_dim). The refiner model uses only OpenCLIP-ViT/G. Each encoder captures different semantic nuances, so prompt engineering that works for one may not optimize the other.

Sources: SDXL model card; Stability AI generative-models README. Once the prompt is settled, the sampler and CFG choices decide how faithfully it lands — covered in samplers and schedulers.

Flux: CLIP + T5 Changes the Game

Flux replaces dual-CLIP with CLIP-ViT/L + T5-XXL. Key differences: Property | CLIP (Flux) | T5 (Flux)

|---|---|---

Tokenization | BPE | SentencePiece Max tokens | 77 | Configurable Strength | Visual-semantic alignment | Language understanding, long context The HFEmbedder class routes to CLIP or T5 based on model version. Flux prompts can be much longer than 77 tokens, but tokenization differs — you cannot copy-paste SD prompts and expect identical token alignment.

Source: Flux conditioner.py HFEmbedder class. Prompt | Effect | Notes

|---|---|---

back lighting / rim lighting | Subject separation, edge glow | Weight ~1.2–1.5; >1.5 may distort faces volumetric lighting | Atmospheric light shafts | Best with visible light source dimly lit / dark atmosphere | Night/moody scenes | Instantly shifts to low-key crepuscular rays / god rays | Dramatic sunbeams | Requires sunlight or explicit source side lighting / oblique lighting | 3D form, texture | 45°–90° from camera Key principle: Any light effect requires a light source; adding light source object prompts produces better results.

Source: Civitai article 1372 (DreamShaper 6/7, SD 1.5).

Common Errors and Fixes

Symptom | Likely Cause | Fix

|---|---|---

Prompt seems ignored after ~75 tokens | Chunk boundary splits subject from modifiers | Use BREAK to control chunking; keep subject + key modifiers in first 75 tokens Faces distort at high emphasis weights | (token:>1.5) amplifies embedding norms | Cap weights at 1.3–1.4; use multiple moderate weights Lighting prompts do nothing | No light source object in scene | Add sunlight, lamp, neon sign, window light Flux ignores (token:weight) | Syntax not supported in Flux pipeline | Use natural language emphasis; test ComfyUI's CLIPTextEncode node SDXL prompt behaves differently than SD 1.5 | Dual encoder concatenation changes balance | Test prompts on both; adjust weighting per encoder

FAQ

Why does moving tokens around change the image in Stable Diffusion?

Token order affects positional embeddings and, past 75 tokens, which chunk a token lands in. Within a chunk, full self-attention means every token can influence every other — but the learned attention patterns may favor certain syntactic positions. The effect is real but not a simple "earlier = stronger" rule.

Does prompt order matter in SDXL vs SD 1.5?

The chunking behavior is identical (both use CLIP-ViT/L with 75-token chunks). SDXL adds a second encoder (OpenCLIP-ViT/G) that processes the same tokens independently. Order effects exist in both, but the dual-encoder concatenation can amplify or dampen them unpredictably.

What text encoder does Flux use for prompting?

Flux uses two encoders simultaneously: CLIP-ViT/L (OpenAI, BPE, 77 tokens) and T5-XXL (Google, SentencePiece, configurable length). Both condition the model; T5 handles longer, more complex descriptions.

Why are some tokens ignored in long Stable Diffusion prompts?

Prompts exceeding 75 tokens are split into chunks. Tokens at chunk boundaries lose cross-chunk attention. Tokens in later chunks only influence the UNet through the concatenated embedding — they never "see" tokens in earlier chunks during CLIP encoding.

How does the 75-token CLIP limit affect my prompts?

Plan your first 75 tokens to contain the subject, key style modifiers, and critical lighting/composition tokens. Use BREAK to force a chunk boundary to control chunking. The SD WebUI supports prompts longer than 150 tokens (three or more chunks) without truncation.

Can I use (token:weight) syntax in Flux?

Unconfirmed. The syntax is implemented in SD WebUI's prompt_parser.py. Flux uses a different inference stack (BFL/ComfyUI). Test in your workflow; do not assume it works.

What's the difference between rim lighting and back lighting?

Both create edge separation. rim lighting emphasizes a thin bright outline; back lighting places the light source behind the subject, often creating a silhouette with edge glow. Use both at moderate weights (1.1–1.3) for stronger separation.

| Source | Contribution

|---|---|---

1 | CLIP repository | BPE tokenizer, context_length=77 2 | SD WebUI Features wiki | Emphasis syntax, 75-token chunking 3 | prompt_parser.py | Emphasis weight parsing, round_bracket_multiplier = 1.1 4 | SDXL model card | Dual encoder architecture 5 | Stability AI generative-models | SDXL encoder usage 6 | Flux conditioner.py | CLIP + T5 routing 7 | Flux repository | Architecture overview


Sources