Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

AXPO (Agent eXplorative Policy Optimization) is a new training method for multimodal agents that use external tools.

It fixes the reasoning prefix, resamples tool calls and continuations, and uses uncertainty to choose better prefixes, reducing failures in tool-using rollouts.

Across nine multimodal benchmarks and three Qwen3-VL-Thinking scales, AXPO beat GRPO-based training.

An 8B model trained with AXPO even surpassed a 32B base model on Pass@4, showing better efficiency and reasoning performance.