Neural networks have also been shown to be powerful generative models for language modelling (Bengio et al., 2003;Mikolov et al., 2010) and machine translation (Kalchbrenner and Blunsom, 2013;Devlin et al., 2014;Sutskever et al., 2014).
currently these models lack awareness of syntax, which limits their ability to include longer-distance dependencies even when potentially unbounded contexts are used.