Perfeature standardization does not preserve cosine similarity, nor does it fix the large macro variance problem.
for the example in figure 1, the paraphrase model may have learned the mapping from "play for" to "whose team is" in a source domain.