Build Recurrent-Depth Transformers with OpenMythos for MLA, GQA, Sparse MoE, and Loop-Scaled Reasoning

The tutorial uses OpenMythos in Google Colab to build recurrent-depth transformer variants with MLA and GQA attention.

It compares the models, inspects the recurrent injection matrix, and checks spectral-radius-based stability.

The models are trained on a synthetic digit-chain sum-modulo task to study loop-based reasoning and modular arithmetic.

The example shows how to reuse one transformer’s parameters for deeper computation and evaluate compact reasoning behavior.