The main motivation for incorporating these multiple "views" of the state sequence is that it removes the need for the RNN cell to accumulate the representation of the whole sentence as it processes the input, and therefore it should have more capacity for modeling local dependencies.
we fix a number r, the number of attention heads, and compute an r ×T matrix A of attention weights α jt , representing the importance of position t in the input for the j th attention head.