QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents

Researchers introduced QUACK, an open-source framework for auditing whether multimodal game-playing agents' spoken claims match what they actually perceive and do in social deduction games.

QUACK reconstructs agents’ true trajectories from game logs and checks each utterance for consistency with evidence, not just for winning performance.

When tested on frontier vision-language models, the system found substantial rates of unsupported spatial claims and accusations.

The work offers a practical way to evaluate grounding, hallucination, and deceptive behavior in multimodal AI systems.