OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents

Researchers introduced OpenWebRL, an open framework for training visual web agents directly on live websites with online multi-turn reinforcement learning.

The system combines live-browser infrastructure, supervised initialization, multimodal context handling, success judging, and policy optimization.

Using it, they trained OpenWebRL-4B with limited initialization data and RL tasks, achieving strong results on Online-Mind2Web and DeepShop.

OpenWebRL outperformed prior open agents and remained competitive with proprietary systems like OpenAI CUA and Gemini CUA.

The authors plan to release the data, models, and code, highlighting a path to better open web agents with less dependence on costly supervision.