DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes

Researchers introduced DecoupleMix, a two-stage framework for optimizing vision-language model training mixtures.

It separates inter-class ratio search from intra-class allocation using quality, difficulty, and diversity signals.

The method outperformed heuristic baselines, scaled well from proxy datasets to larger settings, and improved data mixture design for VLM pretraining.

A VLM trained with 80B extra multimodal continue-pretraining tokens stayed competitive with stronger open-source models.