Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration

Researchers found that joint training of multi-token prediction and reinforcement learning can fail without careful balance.

They introduced an online adaptive coefficient calibration method to tune the mix during training.

The method matches or outperforms the detach baseline across six math reasoning benchmarks.

The work offers a practical way to combine reinforcement learning from verifiable rewards with multi-token prediction in large language models.