WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

Researchers introduced WIDE, a token-level dynamic width pruning method for LLM inference that selectively trims attention-head and FFN-channel groups per token.

The framework is end-to-end differentiable, uses a two-stage training process, and co-designs pruning with execution kernels.

It supports efficient computation in both prefill and decode phases, while retaining accuracy better than prior dynamic pruning methods.

At 50% sparsity, the paper reports measurable speedups, pointing to a more hardware-efficient tradeoff between LLM quality and inference cost.