As far as we know, we are the first to introduce explicit loss functions to optimize the attention.
aRU is designed as an update unit based on current decoding state, aiming to retain the attention on salient parts but weaken the attention on irrelevant parts of input.