To the best of our knowledge, the current widely-used attention mechanism usually ignores the influence of positions when measuring the weights of each word.
we propose a Position-Aware Attention (PAA) model which regularly decreases the weight of word i with respect to the distance between word i and word t. Supposing that we compute the context vector c t at position t, PAA first computes the weight for each word by: where f (s t−1 , h i ) is the score function which computes the weight of h i given previous decoder hidden state s t−1 and the corresponding distance.