Our AGT method adds the right two columns of Figure 1.
1) the transform gate T l now is not a function of y l−1 , but a function of the selection vector s + l , which is determined by the attention distributed to the given input x by the l th layer (will be discussed next), and 2) the function f takes as input the concatenation of y l−1 and s + l to create feature representation y l .