Semantic Generative Tuning for Unified Multimodal Models

Researchers propose Semantic Generative Tuning for unified multimodal models.

The study finds high-level semantic tasks, especially image segmentation, work better than low-level pixel tasks as a bridge between visual understanding and generation.

Using segmentation as a generative proxy improves both multimodal comprehension and image generation quality.

The approach addresses a key training mismatch in unified multimodal AI with a single post-training task.