The ranking of documents can be decomposed into rankings of document pairs as suggested in (Radinsky and Ailon, 2011).
a model's retrieval quality can be examined by checking across a range of individual document pairs, namely, how likely a model can assign a higher score for a more relevant document.