Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards

Researchers introduced CIPO, a policy-optimization method for language models that turns failed RL trajectories into correction-focused supervision.

CIPO extends RL with verifiable rewards by extracting useful learning signals from mistakes instead of discarding failed attempts.

In tests across 11 benchmarks, it outperformed strong baselines on reasoning and self-correction metrics.

The approach improved math reasoning and code generation performance, including pass@K-style results, without relying on external labels.