This is done during embedding learning, in contrast to our post-processing method.
(i) The column normalization constraint (Equation (35)) will align W 0 and W 1 , as we just split the original word embeddings and add them up again.