The analysis of the attention model indicates that the model attends heavily to the contextual history for the translation of some pronouns.
this distribution is then used to compute a new representation of that word: this new representation is set to an expectation (under the attention distribution specific to the word) of word representations from the layer below.