In addition, we have h s,t (2) = t + 1 and h s,t (3) = σ 3(t − T − 1) .
hence we have which shows that the probability otherwise of predicting $ increases over time and eventually (for t 3T ) far outweighs the probability of predicting a. in this case the RNN is consistent (see Lemma 16 in the appendix).