Capturing LLM Capabilities via Evidence-Calibrated Query Clustering

Researchers introduced ECC, a clustering method that uses model comparison evidence to group queries by the LLM capabilities they actually require.

ECC calibrates semantic embeddings with limited posterior comparison data, then models cluster profiles using a Bradley-Terry framework.

The method ranks model capabilities more accurately than human-labeled or embedding-only baselines.

It also improves downstream query routing by better matching tasks to the capabilities they demand.