Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning

Researchers identified a failure mode called futile reasoning, where LLMs produce plausible but incorrect chains of thought on unsolvable tasks.

They traced the problem to capability overreach and miscalibration, which push models to keep reasoning past their limits.

To address this, they proposed CaRL, a reinforcement learning method that uses failed outputs as refusal training data.

CaRL rewards models for refusing when appropriate, aiming to improve reliability and reduce wasted computation.