ComfyUI Acceleration Stack Version Matrix

TL;DR

• xformers on Windows: install from PyPI wheels for PyTorch ≥2.10.0 using pip install -U xformers --index-url https://download.pytorch.org/whl/cu128 for CUDA 12.8, or cu126 for CUDA 12.6. The source claim that xformers needs PyTorch 2.5.1 is outdated; current wheels require 2.10.0.
• Triton on Windows: use the woct0rdho/triton-windows fork, pip install triton-windows==3.3.1.post19, paired with PyTorch 2.9+ and CUDA 12.8. The MSVC v143 toolchain must be on the system PATH.
• SageAttention vs. FlashAttention: SageAttention requires torch≥2.3.0 and triton≥3.0.0; FlashAttention 2 requires torch≥2.2+ and CUDA≥12.0. Both can coexist with xformers in the same environment, but install order matters — xformers first if you want wheels, then SageAttention, then FlashAttention from source last.

For related setup guides, see ComfyUI on Windows from scratch, PyTorch nightly vs stable, and NVIDIA driver compatibility matrix.

The ComfyUI acceleration stack version matrix

The core problem is that each library in the ComfyUI acceleration stack — Triton, SageAttention, FlashAttention, xformers — ships its own binary wheels or build requirements that pin to specific PyTorch and CUDA versions. Mismatching these creates silent failures or compilation errors that can consume an entire afternoon.

Which PyTorch CUDA build do you actually need?

[Source: verified against primary project documentation for each row above]

How to read the matrix

The PyTorch stable channel (2.7.0 as of this writing) ships wheels for CUDA 11.8, 12.6, and 12.8 ^[1]. The nightly channel adds CUDA 12.8 builds for PyTorch 2.9 ^[2]. xformers currently builds wheels only against PyTorch ≥2.10.0, so if you are on PyTorch 2.7 stable you must build from source or use an older xformers release ^[3]. SageAttention supports a wider PyTorch floor (≥2.3.0) and is the most forgiving option for older installs ^[4].

What each library actually does in ComfyUI

What is Triton doing in the attention pipeline?

Triton provides GPU kernels written in a Python-like DSL that compile to custom PTX. In ComfyUI, Triton powers the Triton backend for SageAttention (sageattn_qk_int8_pv_fp16_triton). On Windows, Triton requires the woct0rdho/triton-windows fork because the upstream project did not publish Windows wheels for most versions. The fork ships prebuilt wheels as triton-windows==3.3.1.post19 ^[5].

What does SageAttention replace?

SageAttention is a plug-and-play drop-in for torch.nn.functional.scaled_dot_product_attention (PyTorch's built-in attention). It quantizes QKᵀ to INT8 and accumulates PV in INT8 or FP8, achieving speedups on Ampere (RTX 30xx), Ada (RTX 40xx), and Hopper (H100) GPUs without accuracy loss. Install via pip install sageattention==2.2.0 --no-build-isolation or compile from source with MAX_JOBS set ^[4].

ComfyUI exposes this as the --use-sage-attention launch flag, which routes all attention through the SageAttention kernel ^[6].

What is xformers and why does it need its own CUDA index?

xformers is a library of modular, hackable attention kernels (memory-efficient exact attention, sparse attention, block-sparse attention, fused linear layers) maintained by Meta Research. It ships prebuilt wheels on PyPI but targets specific CUDA toolkits via PyTorch's wheel index (cu118, cu126, cu128, cu130). The install command depends on your CUDA version — using the wrong index URL produces a missing-dependency error at import time ^[3].

Does FlashAttention still matter for ComfyUI?

FlashAttention 2 provides the fastest exact attention kernel for NVIDIA GPUs (Ampere and newer). It compiles from source using the NVIDIA CUDA toolkit and requires the ninja build tool. On Windows, FlashAttention 2 is not officially supported until v2.3.2, and even then compilation requires manual patching of C++ template files to resolve constant-referencing errors in flash_fwd_launch_template.h and flash_bwd_launch_template.h ^[7][8]. For most ComfyUI users on Windows, SageAttention or xformers are the practical choice.

Install order that actually works

The safest installation sequence avoids the cascading rebuilds that occur when you install xformers after PyTorch has already been locked to a specific CUDA toolkit, or when Triton tries to compile against a PyTorch ABI it does not match.

1. Install PyTorch first, with CUDA 12.8

pip install torch==2.9.0.dev20250716+cu128 torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Nightly builds provide the latest CUDA 12.8 compatibility. For stable, use PyTorch 2.7.0 with --index-url https://download.pytorch.org/whl/cu128 ^[1].

2. Install xformers next (if using wheels)

pip install -U xformers --index-url https://download.pytorch.org/whl/cu128

Installing xformers here, before other attention libraries, ensures PyTorch's CUDA context is established correctly. If you skip this step and install from source later, you risk ABI mismatches ^[3].

3. Install Triton Windows fork

pip install triton-windows==3.3.1.post19

The official Triton project does not publish Windows wheels; the woct0rdho fork does. This fork is a mirror of the upstream Triton with Windows-specific build patches ^[5].

4. Install SageAttention

pip install sageattention==2.2.0 --no-build-isolation

SageAttention compiles from source on install. The --no-build-isolation flag prevents pip from creating an isolated build environment that would duplicate the PyTorch you just installed ^[4].

5. FlashAttention 2 last (optional, Windows)

git clone https://github.com/Dao-AILab/flash-attention
cd flash-attention
pip install ninja
MAX_JOBS=4 pip install flash-attn --no-build-isolation

On Windows, expect compilation failures unless you apply the template patches described in the errors section below ^[7][8].

Common errors and fixes

Symptom: ModuleNotFoundError: No module named 'triton' after pip install on Windows

Cause: the system PATH is missing the Visual Studio MSVC compiler directory. Triton's Windows build requires C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat in the environment, or the MSVC bin directory added to PATH manually ^[5].

Fix: run vcvars64.bat before activating your ComfyUI virtual environment, or add the MSVC Hostx64\x64 directory to the system PATH and restart your terminal.

Symptom: FlashAttention 2 fails to compile on Windows with error C2975 in flash_fwd_launch_template.h

Cause: the C++ template code references a variable Headdim that MSVC's compiler rejects as a non-type template parameter in certain contexts ^[8].

Fix: replace Headdim identifiers in template instantiations with their concrete integer values (e.g., Flash_bwd_kernel_traits<128, 64, 128, ...> instead of Flash_bwd_kernel_traits<Headdim, ...>). The full patched files are available in GitHub issue #595 ^[8].

Symptom: Triton fails at runtime with a setuptools conflict

Cause: recent versions of setuptools use pkg_resources in a way that conflicts with Triton's build system on Windows ^[9].

Fix: downgrade setuptools: pip uninstall -y setuptools && pip install setuptools==78.1.1.

Symptom: torch.cuda.is_available() returns False after installing a CUDA 12.8 build

Cause: the NVIDIA driver on your GPU is too old to support CUDA 12.8, or the CUDA toolkit version in your PATH does not match the PyTorch build.

Fix: verify your GPU supports CUDA 12.8 (RTX 30xx and newer do; GTX 10xx does not), and ensure nvcc --version reports a compatible CUDA toolkit version. PyTorch wheels embed their own CUDA runtime, but some Triton operations require the toolkit on the system path ^[1][5].

FAQ

What version of PyTorch works with FlashAttention on Windows?

FlashAttention 2 requires PyTorch 2.2 or later and CUDA 12.0 or later. On Windows, FlashAttention is not officially supported until v2.3.2, and community reports confirm compilation on Windows 11 with PyTorch 2.2+cu121 and CUDA 12.2, but only after applying template patches for MSVC compatibility ^[7][8].

Can I use Triton, SageAttention, and xformers together?

Yes, but they serve different roles and ComfyUI routes attention through one backend at a time. SageAttention (--use-sage-attention) and FlashAttention (--use-flash-attention) are mutually exclusive in the CLI. xformers can operate alongside either but is disabled by --disable-xformers ^[6]. The practical recommendation is to pick one primary attention backend (SageAttention for Windows, xformers for Linux) and use Triton as the kernel compiler layer for SageAttention only.

What version of xformers works with my PyTorch version?

xformers ships prebuilt wheels for PyTorch ≥2.10.0 with CUDA 11.8, 12.6, 12.8, and 13.0 ^[3][10]. Older xformers releases support older PyTorch versions — for example, xformers 0.0.31 requires PyTorch 2.7.1, and xformers 0.0.30 requires PyTorch ≥2.7 ^[10]. If you are on PyTorch 2.7 stable, you must either build xformers from source or pin to an older release. Using the wrong PyTorch version with xformers wheels results in a missing-dependency error at import time ^[3].

What GPU do I need for the full acceleration stack?

NVIDIA GPUs starting with the RTX 30xx series (Ampere) are the practical minimum. SageAttention achieves its best speedups on RTX 40xx (Ada) and H100/H20 (Hopper). FlashAttention 2 supports Ampere, Ada, and Hopper GPUs. Turing GPUs (RTX 20xx, T4) are not supported by SageAttention, but FlashAttention maintains a separate flash-attention-turing repo with a core subset of features for Turing ^[7]. GTX 10xx (Pascal) is not supported by either FlashAttention 2 or SageAttention.

What does --fast fp16_accumulation actually do in ComfyUI?

It toggles the fp16_accumulation performance feature, which uses fp16 rather than fp32 for intermediate accumulation in attention and matrix-multiply nodes. This reduces VRAM usage and improves throughput on NVIDIA GPUs, at the cost of minor numerical precision. ComfyUI's CLI defines it as an untested optimization under --fast ^[6].

Sources

• Civitai Article 9555 — Installing Triton and Sage-Attention Flash-Attention and xformers — Windows-focused installation guide with version claims (some outdated, see corrections)
• Civitai Article 17252 — Installation Guide ComfyUI Torch2.9 Triton3.3 SageAttention fp16 accumulation — Manual install steps for PyTorch nightlies and specific Triton/SageAttention versions
• Civitai Article 3656 — Resource monitor for ComfyUI — Context for ComfyUI ecosystem; not directly relevant to stack installation
• xformers GitHub README — Current install commands, CUDA index URLs, PyTorch 2.10.0 minimum requirement
• xformers CHANGELOG — Version-to-PyTorch compatibility mapping across 0.0.28–0.0.35
• PyTorch Get Started (official) — Current stable (2.7.0) and nightly channels with CUDA 12.6/12.8 Windows wheels
• PyTorch Previous Versions — Historical CUDA version mapping (cu124, cu126, cu128) for each PyTorch release
• SageAttention GitHub README — Python/PyTorch/CUDA requirements, v1/v2/v2++/v3 version matrix, install commands
• FlashAttention GitHub README — PyTorch ≥2.2, CUDA ≥12.0 requirements, Windows support note
• ComfyUI cli_args.py — --use-sage-attention, --use-flash-attention, --disable-xformers, --fast flags and PerformanceFeature enum
• ComfyUI requirements.txt — torch unpinned in current main branch
• FlashAttention issue #595 — Windows compilation fixes for Headdim and kBlockM template errors in MSVC