Qwen Team Releases FlashQLA: a High-Performance Linear Attention Kernel Library That Achieves Up to 3× Speedup on NVIDIA Hopper GPUs

QwenLM released FlashQLA, an MIT-licensed GPU kernel library for Gated Delta Network linear attention.

Built on TileLang, it replaces slower Triton kernels with Hopper-specific optimizations for forward and backward passes.

In tested scenarios, FlashQLA delivers roughly 2–3× faster forward execution and about 2× faster backward execution.

The upgrade targets a major bottleneck in long-context LLM workloads, especially on NVIDIA Hopper GPUs like H100 and H200.

It could lower latency and compute cost for Qwen models such as Qwen3.5 and Qwen3.6, as well as other Hopper-based deployments.