New NVIDIA Research Shows Speculative Decoding in NeMo RL Achieves 1.8× Rollout Generation Speedup at 8B and Projects 2.5× End-to-End Speedup at 235B

NVIDIA added speculative decoding to NeMo RL v0.6.0 to speed up RL rollout generation.

The approach uses a draft model plus verifier path, preserving the target model’s exact output distribution.

It delivered a 1.8× generation speedup at 8B scale and could reach about 2.5× end-to-end at 235B.

Because rollouts are the main RL post-training bottleneck, this can cut training time and cost without off-policy fixes.