Every Component is a Lookup: Token Attribution and Composition from a Single Decomposition

Researchers introduced Unpack, a non-invasive interpretability method for transformers based on a single forward pass.

Unpack backpropagates credit through attention and MLP sublayers to assign token-level attribution and trace K/Q/V interaction paths.

The method was validated on GPT-2 small and Pythia, recovering known indirect object identification circuits and duplication-suppression patterns.

It offers a way to study how transformer components combine predictions without gradients, interventions, or labeled circuit ground truth.