LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?

A study of BrowseComp found many LLM search agents lean on memorized knowledge instead of external evidence, sometimes answering without using tools.

When supporting evidence was removed, performance often worsened, suggesting some systems are verifying what they already know rather than searching deeply.

To address this, the authors introduced LiveBrowseComp, a benchmark of 335 human-written questions based on facts from the last 90 days.

In tests, evaluated agents scored below 2% in closed-book settings and dropped sharply even with search assistance, highlighting the difficulty of genuine recent-fact search.