Following notations in Section 2.3, β t,i ∝ exp(score(d t , row i )) obtains the attention weight with respect to each row.
we keep most recent history information sequence within history window and denote them as hist(r i,j ).