Toward Native Multimodal Modeling: A Roadmap

The paper lays out a roadmap for native multimodal modeling, where one transformer natively handles multiple modalities for understanding and generation.

It defines native multimodal modeling and organizes existing systems by input-output structure, including late, mid, and early fusion approaches.

It also maps key industrial concerns for building, training, deploying, and evaluating unified multimodal transformers.

The core takeaway is a shift away from stitched modality pipelines toward integrated architectures that better support flexible multimodal reasoning.