In ATR, the role of q t is similar to that ofh t in GRU (see Equation 4).
we completely wipe out the recurrent non-linearity Figure 2: Visualization of the difference between σ(x + y) (the input gate) and σ(x − y) (the forget gate).