NVIDIA AI Unveils ProRL Agent: A Decoupled Rollout-as-a-Service Infrastructure for Reinforcement Learning of Multi-Turn LLM Agents at Scale

nVIDIA researchers introduced ProRL AGENT, a scalable infrastructure for reinforcement learning (RL) training of multi-turn LLM agents.

Its Rollout-as-a-Service design decouples rollout orchestration from the training loop and resolves resource conflicts between I/O‑intensive environment interactions and GPU‑intensive policy updates that bottleneck agent development.

ProRL AGENT runs as a standalone HTTP service managing the full rollout lifecycle; the RL trainer interacts with it via an API.

A three-stage asynchronous pipeline (INIT, RUN, EVAL) maximizes throughput by overlapping jobs: INIT spins up sandbox containers and tools; RUN drives multi‑turn agent loops and collects trajectories; EVAL scores results against ground truth to produce reward signals.

The system uses Singularity for sandboxing and rootless execution.