Both of Long-Short Term Momery (LSTM) (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Cho et al., 2014) perform much better than the standard tanh RNN.
sentence Encoder that encodes lead-in sentence, sentences in permanent memory and answer candidates.