Our proposed method breaks the limitation that masked n-grams and predicted ones have to be the same length during BERT training.
according to IBM Model 2, given a source n-gram x l 1 and a target one y m 1 , where l and m are the numbers of tokens in the source and target n-grams respectively, the translation probability from x l 1 to y m 1 is calculated as: where = p(m|x l 1 ), i.e.