We hypothesize that this is because LEXSUB learns the lexical relations in a linear subspace which happens to be the simplest possible manifold.
this embedding layer is initialized with the embedding that is to be evaluated and is not fine-tuned during training.