As we see, the introduction of our method to the standard backbone model (Trans.-Base) leads to substantial improvements across the validation and test sets.
language models are still important in reducing the size of the candidate set, regularizing word embeddings and generating fluent sentences.