LionMuon: Alternating Spectral and Sign Descent for Efficient Training

Researchers introduced LionMuon, a new optimizer that alternates Lion and Muon updates on a fixed schedule while sharing momentum.

The design lowers average iteration cost and optimizer memory by combining sign-based and spectral matrix-sign updates.

Across experiments, LionMuon outperformed or matched strong baselines such as AdamW, Lion, Signum, and Muon on multiple model sizes and datasets.

The paper also provides complexity bounds showing when this alternating approach is theoretically optimal, especially under heavy-tailed noise.