Skill-Conditioned Gated Self-Distillation for LLM Reasoning

Researchers introduced Skill-Conditioned Gated Self-Distillation, a new method for improving LLM reasoning with weaker supervision.

It retrieves skill-mistake pairs, builds multiple skill-conditioned teachers, and uses a verifier plus gated loss to keep useful disagreements while filtering noisy signals.

On math reasoning benchmarks such as AIME24, AIME25, and HMMT25, it outperformed GRPO and compared strongly with answer-conditioned self-distillation.

The approach delivered an average 6.2% improvement over GRPO on Qwen3-1.7B, suggesting better reasoning without fully trusted reference answers.