The first two steps have been shown to be beneficial in previous work (Artetxe et al., 2016), while the second length normalization guarantees the final embeddings to have a unit length.
the dot product of any two embeddings is equivalent to their cosine similarity and directly related to their Euclidean distance 1 , and can be taken as a measure of their similarity.