AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models

Researchers introduced AgentGrounder, a zero-shot framework for grounding natural-language object queries in 3D point clouds using multimodal language models.

The pipeline first builds an object lookup table from colored point clouds, then an online agent retrieves candidates, scores geometry, and renders views only when needed.

On ScanRefer and Nr3D, AgentGrounder outperforms SeeGround, showing stronger zero-shot 3D visual grounding without task-specific 3D training.

The approach improves localization by combining selective retrieval, geometric reasoning, and on-demand visual inspection.