This is indeed a pretty simple trick, and per-feature standardization (with zero mean) is also a standard data preprocessing method.
it is not self-evident that this kind of standardization shall be applied on pre-trained word embeddings before using them in deep neural networks, especially with the obvious downside of rendering the word embedding algorithm's loss function sub-optimal.