Our proposed method can give the model more explicit and strong cross-lingual training signals so that the pre-trained model contains much beneficial cross-lingual information for unsupervised machine translation.
we can further improve the translation performance significantly, compared with Lample and Conneau (2019) (with the significance level of p<0.01).