Grid Games: The Power of Multiple Grids for Quantizing Large Language Models

Researchers propose power-of-two grids, letting each quantization group choose among multiple 4-bit formats using scale bits.

The theory shows the biggest gains come with small groups, while benefits fade for very large groups.

Tests across PO2(NF4), MPO2, PO2(Split87), and SFP4 show adaptive grids beat single-grid FP4.

The method improves both post-training quantization and pre-training for weights alone and weights plus activations.