Previous approaches benefit mostly from crosslingual n-gram embeddings, but recent work proves that cross-lingual language model pretraining could be a more effective way to build initial unsupervised machine translation models (Lample and Conneau, 2019).
in their method, the cross-lingual information is mostly obtained from shared Byte Piece Encoding (BPE) (Sennrich et al., 2016b) spaces during pre-training, which is inexplicit and limited.