OpenAI Baselines: ACKTR & A2C

OpenAI Baselines highlights ACKTR as a reinforcement-learning method that can outperform A2C on Q-Bert.

ACKTR uses natural-gradient updates with KL control, aiming to improve sample efficiency over standard first-order methods.

The method is based on KFAC and delivers better learning efficiency with only a moderate increase in computation.