The recurrent model makes decoding harder than nonrecurrent neural network language models.
we use RNNLM because the model outperforms other NNLMs (Mikolov, 2012) and the result suggests that the model is more likely to capture semantic plausibility.