InstructSAM: Segment Any Instance with Any Instructions

Researchers introduced InstructSAM, a unified framework for instruction-driven multi-instance segmentation.

It uses learnable instance queries and hybrid attention to turn text instructions and visual context into detector queries for SAM3.

The system enables efficient one-pass mask prediction for multiple instances, rather than sequential processing.

They also released Inst2Seg, a dataset for training and evaluating instruction-based segmentation.

Results show stronger benchmark performance and better efficiency than prior methods.