Since a typical user will eventually process all received mail, we assume that producing a quality ranking will more directly measure the impact on the user than accuracy or F1.
we focus on ROC curves and area under the curve (AUC) since both reflect the quality of the ranking produced.