Recurrent neural networks (RNNs) have been successful for modeling sequences (Elman, 1990).
rNNs equipped with internal short memories, such as long short-term memories (LSTM) (Hochreiter and Schmidhuber, 1997) have achieved a notable success in sequential tasks (Cho et al., 2014;Vinyals et al., 2015).