Once the attention is defined, the representation of the input is constructed as The problem with this traditional attention model is that the final hidden state is a function of all the inputs, hence it is usually more "informative" than the earlier hidden states due to semantic accumulation (Wang et al., 2016).
most of the attention signal is assigned to the hidden states toward the end of a sequence.