Also, notice that α = 0.99 brings the SCRN's results closer to the LSTM's results on non-English data.
we think LSTM stores too much of the long-term memory via its trainable forget gate, while in SCRN we can directly control this through the α.