Active Learners as Efficient PRP Rerankers

Researchers reframed pairwise ranking prompting as active learning over noisy pairwise judgments.

They introduced active rankers that improve reranking quality when LLM calls are limited.

A randomized-direction oracle removes systematic position bias while using only one call per pair.

The approach delivers better top-K ranking quality per call, including stronger NDCG@10 in constrained settings.