PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers

Researchers introduced PRISM, a benchmark for evaluating AI peer reviewers across depth, novelty assessment, flaw detection, and constructiveness.

PRISM uses argument mining and retrieval-based verification to score review quality more rigorously than simple text comparisons.

Testing five automated systems and human reviews from ICLR, ICML, and NeurIPS showed LLMs can rival humans on some dimensions.

But no system consistently matched human reviewers across all criteria, highlighting the limits of current LLM peer reviewers.