Peng's Q($\lambda$) for Conservative Value Estimation in Offline Reinforcement Learning

Researchers introduced CPQL, a conservative multi-step offline RL method built on Peng’s Q(λ).

The method replaces the standard Bellman operator with a conservative value estimator and proves near-optimal guarantees.

CPQL outperforms prior offline single-step baselines on D4RL benchmarks.

It also provides a more stable starting point for later online fine-tuning after offline pretraining.