DynMuon: A Dynamic Spectral Shaping View of Muon

Researchers introduced DynMuon, a dynamic variant of the Muon optimizer that changes its spectral emphasis during training.

The exponent is scheduled from positive to mildly negative based on curvature, stochastic noise, and training stage.

In experiments, DynMuon reached lower validation loss with 10.6% to 26.5% fewer steps than Muon.

The result suggests adaptive spectral shaping can improve convergence and make training more efficient.