We promote output diversity by training f para on an aggressivelyfiltered subset of PARANMT-50M (Wieting and Gimpel, 2018), a large corpus of backtranslated text.
we apply three filters: (1) removing sentence pairs with more than 50% trigram or unigram overlap to maximize lexical diversity and discourage copying; (2) removing pairs with lower than 50% reordering of shared words, measured by Kendall's tau (Kendall, 1938), to promote syntactic diversity; and (3) removing pairs with low semantic similarity, measured by the SIM model from Wieting et al. (2019)