It contains an additional sublayer which performs multi-head attention over the outputs of the encoder block.
decoding layer d l i is the result of a) multi-head attention over the outputs of the encoder which in turn is a function of the encoder memory and the outputs from the previous layer: where, the keys and values are the encoder outputs and the queries correspond to the decoder input, and b) the multi-head self attention which is a function of the generated outputs from the previous layer: Deliberation networks Xia et al., 2017) build on the standard sequence to sequence architecture to add an additional decoder block (in our case, with 3 layers -see Figure 2).