Evaluating a single translation Most of the MT evaluation metrics are not designed to do pairwise, but absolute evaluation.
we are interested in generating a score for a single translation t 1 given a reference r. To achieve this with our pairwise network, we compute the goodness margin, which tells us how good translation t 1 is better than any other translation t ∅ given the reference r. In this case, t ∅ is the average representation for all translations observed during training.