RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility States

Researchers introduced RoMeRL, a new memory-learning method for self-evolving LLM agents.

RoMeRL compresses trajectory utilities into fixed-size, task-level memory states organized by outcome polarity and memory dynamics.

On ALFWorld and LifelongAgentBench, it improved performance, increased feedback density, reduced memory size, and cut LLM calls.

The approach also helps prevent the memory-reward trap by limiting erroneous reinforcement of irrelevant memories.