Specifically we consider the following approaches to language modelling: KenLM We use KenLM (Heafield et al., 2013) to build conventional (word) n-gram language models with modified Kneser-Ney smoothing, for differing orders of n. char-rnn We consider a character-level LSTM language model.
we use a TensorFlow implementation 13 of char-rnn, 14 with its default parameter settings.