Researchers from MIT, NVIDIA, and Zhejiang University Propose TriAttention: A KV Cache Compression Method That Matches Full Attention at 2.5× Higher Throughput

Researchers from MIT, NVIDIA, and Zhejiang University introduced TriAttention, a KV cache compression method for long-context LLM reasoning.

Unlike prior eviction methods that rank tokens in post-RoPE space, TriAttention uses pre-RoPE Q/K concentration to better decide which cached tokens to keep.

On AIME25 with 32K-token generation, it matched full-attention accuracy while delivering 2.5× higher throughput or using 10.7× less KV memory.

The approach could make long reasoning runs more practical by reducing GPU memory pressure without sacrificing answer quality.