Implicit Hierarchical GRPO: Decoupling Tool Invocation from Execution for Tool-Integrated Mathematical Reasoning

Researchers introduced IH-GRPO, a training method that separates tool-use decisions from tool execution in large language models.

The paper formalizes delayed execution with hierarchical control and derives a surrogate loss for implicit hierarchical policy learning.

This design aims to reduce disruptions to reasoning flow, addressing a known weakness in tool-integrated LLM reasoning.

Experiments show improved performance over strong baselines across multiple math benchmarks and model sizes, including Qwen3-1.7B, Qwen3-4B, and Qwen3-8B.