Leveraging much more monolingual data from Wikipedia, their work shows a big potential of pre-training for unsupervised machine translation.
the cross-lingual information is obtained mostly from the shared BPE space during their pre-training method, which is inexplicit and limited.