Relevance is not answerability: six signals, and none of them beat plain cosine

A benchmark study tested six gating signals for deciding whether to answer or abstain on 500 LongMemEval questions.

Dense top-1 cosine, reranker, hybrid, entailment, margin, and ratio signals all struggled to separate answerable from unanswerable cases.

The best AUC was still the existing relevance signal, showing that retrieval can find related context without proving answerability.

Even stronger cross-encoder retrieval worked well for relevance, but still could not reliably distinguish answerable questions from merely related ones.