LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

Researchers introduced LongAttnComp, a long-context adaptation of AttnComp for efficient reasoning over very long inputs.

It uses lightweight attention fine-tuning, token-level chunking, token-budget top-p selection, positional reordering, and a two-stage training recipe.

The method performs strongly on long-context benchmarks, matching or beating full-context results on Code-Debug and improving multi-document reasoning transfer across model families.

By compressing inputs more efficiently, LongAttnComp can reduce inference cost while preserving accuracy on demanding tasks like code debugging and multi-document reasoning.