As expected, vectors from this method had near zero accuracy in the semantic portion, but scored well on the syntactic portion, showing that most of these analogies can be answered with morphology alone.
we conclude that the word representation layer in deep biLMs is only faithfully encoding morphology with little semantics.