On the contrary, the other two models distribute attention more evenly among all previous words.
the self-attentive RNN uses a longer context than the self-attentive residual connections but, as the performance on BLEU score shows, this fact does not necessarily mean better translation.