In LSTM, gating mechanism is used to control the information flow such that gradient vanishing problem in vanilla RNN is better handled, and long range dependency is better captured.
as empirically verified by previous works and our own experiments, to obtain fairly good results, training LSTM RNN needs carefully designed optimization procedure (Hochreiter et al., 2001;Pascanu et al., 2013;Dai and Le, 2015;Laurent et al., 2015;He et al., 2016;Arjovsky et al., 2015), especially when faced with unfolded very deep architectures for fairly long sequences (Dai and Le, 2015).