Learning from Language Feedback via Variational Policy Distillation

Researchers introduced Variational Policy Distillation, a variational EM-style training loop for learning from language feedback such as compiler errors, critiques, and self-corrections.

The method co-trains a teacher and student with alternating optimization, letting models learn from rich failure signals instead of only pass/fail rewards.

On open models like Qwen3 and Llama-3.1, it outperformed GRPO and self-distillation on code generation and scientific reasoning benchmarks.

It improved training stability and used no extra memory, though standard RL still did better on some strict math tasks.