In the original BLEU or NIST formulation of the test set unit (or document or system level) evaluation, n-gram matches are computed at the utterance level, but the mean of n-gram matches is computed at the test-set level.
considering the characteristics of the translation paired comparison method, the average of the utterancelevel scores might be more suitable.