We measure performance using accuracy in the top-k ranked translations.
we define top-k accuracy over some set of ranked lists L as follows: where I lk is an indicator function that is 1 if and only if a correct item is included in the top-k elements of list l. top-k accuracy is the proportion of ranked lists in a set of ranked lists for which a correct item is included anywhere in the highest k ranked elements.