MEMO: A Modular Framework for Training a Dedicated Memory Model on New Knowledge Without Modifying LLM Parameters

Researchers introduced MEMO, a modular “Memory as a Model” system that updates LLM knowledge without changing the base model’s weights.

MEMO trains a dedicated MEMORY model on synthesized question-answer data from a document corpus, while a frozen EXECUTIVE LLM handles reasoning through a structured multi-turn protocol.

This separation lets systems absorb fresh information without expensive retraining or catastrophic forgetting, preserving existing capabilities.

The approach also improves cross-document reasoning compared with standard retrieval-augmented methods, with strong results across tested models and benchmarks.