Firstly, even if word analogies as described by Equation (4) existed in the data, it would only be a matter of chance that applying PCA to the entire dataset would recover even slightly these parallel (analogous) word relations visually.
there is no reason to believe that the line through the words in a pair is not almost perpendicular to the surface they are mapped to.