Pre-training has proven to be effective in unsupervised machine translation due to its ability to model deep context information in cross-lingual scenarios.
the crosslingual information obtained from shared BPE spaces is inexplicit and limited.