In contrast to the first attention function, which makes use of the hidden vector s t , the second one is based only on the previous word representations, therefore, it is independent of the current prediction representation.
the normalization of this function still depends on t. To compare our approach with similar studies, we adapted two representative self-attentive networks for application to NMT.