In the datasets that we use, we have labels indicating which QAs are related/relevant to a particular question.
the quality of the scoring function can be evaluated using traditional information retrieval metrics (Robertson and Zaragoza, 2007) such as Precision, MAP, MRR, and NDCG when measured against such labellings.