We find that only sharing Transformer layers is not enough for forcing hidden representations from different languages into a common space, as suggested in experiments (Section 3.4).
we propose three methods to build crosslingual representations based on identical strings, mean/variance, and average linkage.