How LoRA Remembers? A Parametric Memory Law for LLM Finetuning

A new arXiv study treats LoRA as a probe for exact parametric memory in LLM fine-tuning.

The authors find a power-law link between loss reduction, effective parameters, and sequence length.

They also identify a token-level threshold that predicts verbatim recall under greedy decoding.

To improve retention, they introduce MemFT, which shifts training toward harder tokens.