DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization

Researchers introduced DRIFT, a training framework for multi-turn language model optimization.

DRIFT decouples data collection from optimization by sampling offline trajectories from a fixed reference policy.

It uses return-based importance weights during supervised fine-tuning instead of repeated online RL rollouts.

The method is reported to match or outperform multi-turn RL baselines at lower training cost.