Let tp i be the number of test essays correctly labeled as positive by error e i 's binary classifier b i ; p i be the total number of test essays labeled as positive by b i ; and g i be the total number of test essays that belong to e i according to the gold standard.
then, the precision (P i ), recall (R i ), and F-score (F i ) for b i and the macro F-score (F) of the combined system for one test fold are calculated by the macro F-score calculation can be seen as giving too much weight to the less frequent errors.