Current unsupervised adversarial approaches show that it is possible to build a mapping matrix that aligns two sets of monolingual word embeddings without high quality parallel data, such as a dictionary or a sentence-aligned corpus.
(2018) and Chen and Cardie (2018), when setting β to less than 0.01, the orthogonalization usually performs well.