CommandLM: Data-Driven Behavior-Level Descriptor for Ego Vehicles

Researchers introduced CommandLM, a multimodal model that converts fused LiDAR and camera data into short, human-readable descriptions of autonomous driving behavior.

Trained on the new CommandLM-nuScenes dataset, it outperformed a BLIP-2 baseline on captioning metrics and was rated accurate and rule-compliant in most human evaluations.

The authors released the code and dataset, aiming to make autonomous vehicle decisions easier to inspect, validate, and supervise for safety and planning.