It suggests regularities in the semantic space, especially because orthogonal matrices preserve cosine similarity -if M N is orthogonal, two words x, y and their projections π N (x), π N (y) will have the same similarity measure, which is semantically reasonable.
matrices trained by vecUD are only orthogonal for three UD relations, namely conj, dep and appos.