IndexCache, a new sparse attention optimizer, delivers 1.82x faster inference on long-context AI models

IndexCache removes up to 75% of redundant computation in sparse-attention models.

At that context length, it achieved up to 1.82× faster time-to-first-token and 1.48× faster generation throughput.

The approach targets DSA models built on the DeepSeek Sparse Attention architecture, including recent DeepSeek and GLM families.

Preliminary tests included the 744-billion-parameter GLM-5 model.

Because self-attention cost grows quadratically with sequence length, IndexCache can help enterprises deliver faster production-scale long-context user experiences.