Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models

Researchers introduced Chartographer, a framework for generating counterfactual chart variants from reconstructed executable code.

It evaluates chart QA vision-language models with the same questions but changed underlying data to test sensitivity and generalization.

The setup reveals hidden weaknesses when models depend on shortcuts or surface patterns instead of robust visual reasoning.