Updating ComfyUI to version v0.45.0 doubles the speed of local text generation on Qwen3.x family models: the same checkpoints on the same graphics card output up to twice as many tokens per second. The gains come from a rewritten generation hot path, adaptive Tree MTP, and Comfy-Kitchen INT8 attention, and are enabled by a simple client update without changing hardware. For visual pipelines, this means text steps are no longer a bottleneck: local generation on consumer GPUs approaches the speed of cloud APIs.
What happened
In the ComfyUI v0.45.0 release, the generation hot path for Qwen3.x family LLMs has been rewritten, so identical checkpoints on identical cards show a multiple increase in tokens per second. According to the authors' measurements, on an RTX 5090, the Qwen3.5 9B model accelerated from 106 to 219 tokens/s, Qwen3.8 27B when describing a 1536×1536 image accelerated from 55 to 122 tokens/s, and on an RTX 5080, the 9-billion-parameter Qwen3.x accelerated from 91 to 129 tokens/s. The first acceleration mechanism is an optimized GEMV kernel: interleaving of weights and scales, MMA instructions, and prologue fusion with RMSNorm and SwiGLU, tuned to the order of weight consumption. The second is adaptive Tree MTP: instead of a single chain of speculative guesses, a tree up to seven tokens deep is used, the shape of which adapts to the text, where depth is beneficial for code and width for creative writing. The third is Comfy-Kitchen INT8 attention, enabled by the --use-ck-attention flag, which reduces the KV-cache and accelerates pre-fill. According to the authors, generation quality remains unchanged.
Context
Local decoding speed is limited not by computational power, but by video memory bandwidth: to output each token, the GPU must fully read the model weights, and for a 27B model in W4A8 format, this is approximately 14 GB per step. Therefore, gains are sought not in new architectures, but at the level of kernels, speculative decoding, and cache quantization. Tree MTP here is a development of the known family of speculative decoding methods, where the model proposes guesses in advance and verifies them in parallel, while cache quantization is a standard technique that allows long contexts, which previously did not fit in VRAM, to run on the same cards. In the ComfyUI ecosystem, where models are applied within visual pipelines, text stages have historically lagged behind graphical ones, and LLM nodes remained a slow add-on. An important detail: the release does not add a new model and does not change checkpoints; the entire difference is created at the level of executable code.
Why this matters for the industry
For the industry, this is a shift in unit economics: the acceleration comes for free, on the same hardware and same checkpoints, so the marginal cost of text operations -- prompt enhancement, image descriptions, storytelling -- drops to near zero. At 100–220 tokens/s on consumer GPUs, local LLM nodes transform from a slow add-on into a standard building block for products: text stages are executed interactively, without the cloud, API keys, or data transmission outside. Startups lose the basis for selling "API access" for such operations, and value shifts to pipeline orchestration. If the gains are confirmed by independent measurements, expect more nodes and templates where the LLM is a standard pipeline step from description to variations and one-click rerolls, the emergence of chat-like UX patterns within generative editors, and pressure on third-party local inference backends. In the horizon of a couple of years, such a release shows that the competitiveness of local models is determined not only by checkpoint architecture but also by the quality of inference engineering for specific hardware, and the gap between consumer GPUs and the cloud for single-user scenarios is shrinking.
Why this matters for users
If you generate text in ComfyUI using Qwen3.x models, you only need to update to v0.45.0 and set mtp=auto in the Text Generate node: no rebuilding is required, and it works on NVIDIA and AMD under Windows and Linux. For long contexts that previously did not fit in video memory, a separate flag --use-ck-attention is provided, but it is best to try it as a separate experiment with quality control. Practically, this means that text steps no longer slow down the pipeline, and local text functionality can be incorporated into projects as interactive today, while continuing to work with your own checkpoint, chat, and files on your own machine.
What is currently unknown / limitations
The claim of "same quality" is not yet a measured fact: the publication does not include a quality evaluation methodology, and the rewritten hot path changes numerics due to INT8 attention, prologue fusion, and INT8 KV-cache. Without specific evaluations and metrics, this remains a declaration by the authors, so speed should be checked together with output quality on your own tasks. The gain from Tree MTP depends on the nature of the text and the proportion of accepted guesses, so in real tasks it may differ from benchmarks. The stated speed figures were obtained on the authors' configurations; independent measurements are not yet available.
Sources
Author
Look at AI, editorial team