Although word embeddings produced by these models have been shown to give improvements in a variety of downstream tasks, it still remains difficult for these models to learn syntactic word representations owing to their insensitivity to word order.
word embeddings produced by these order-insensitive models are thus suboptimal for syntax-related tasks such as parsing (Andreas and Klein, 2014).