Among the RNN variants, LSTMs have been shown to provide state-of-the-art performance for several benchmark language modeling tasks (Józefowicz et al., 2016;Merity et al., 2018;Melis et al., 2018).
the LSTM with projection introduced by Sak et al.