Recurrent Neural Network LMs composed of LSTM units, especially those augmented with an external memory, have achieved state-of-theart results.
these models still struggle to process long sequences which are more likely to contain long-distance dependencies because of information fading and a bias towards more recent information.