For inference, we found that top-k random sample decoding (Fan et al., 2018) provides the best results for all the systems.
at each decoding step, a token was drawn from the k most likely candidates according to the distribution over the vocabulary.