See What I Mean: Aligning Vision and Language Representations for Video Fine-grained Object Understanding

Researchers introduced SWIM, a training strategy for fine-grained video object understanding.

SWIM uses mask supervision and the NL-Refer dataset to better align language with the intended object.

The method corrects cross-modal attention misalignment so models can localize objects from text alone.

This reduces reliance on explicit visual prompts at inference and improves object identification from language.