OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under Optimal Squared Error Quantization

Researchers introduced OCTOPUS, a deterministic KV-cache codec for transformers that compresses cache data with octahedral triplet quantization.

It jointly quantizes rotated key triplets and uses squared-error-optimized bit allocation to improve compression efficiency.

Across text, video, and audio, OCTOPUS outperforms prior rotation-based methods such as TurboQuant and PolarQuant.

The method reduces KV-cache memory bandwidth and footprint without adding decode-time bandwidth or latency, making long-context inference more practical.