RoboWits: Unexpected Challenges for Robotic Creative Problem Solving

Researchers introduced RoboWits, a new benchmark for testing robotic reasoning and creative problem solving in unexpected situations.

It uses an automated pipeline to create 30 seed tasks and 208 mutated tasks spanning geometry, material, and assembly challenges.

Pre-trained vision-language-action models improved on seed tasks after fine-tuning, but struggled badly on mutated tasks.

The results suggest many robot policies remain fragile when they must adapt, reason, or use tools creatively under novel conditions.