In the last few years, neural networks have become increasingly popular in NLP applications.
neural language models (Bengio et al., 2003;Mnih and Hinton, 2007;Collobert and Weston, 2008) have demonstrated impressive performance at the task of language modeling.