In addition to two sub-layers in each decoder layer, the decoder inserts a third sublayer D l d to perform attention over the output of the encoder stack H L e : where Multi-layer network can be considered as a strong feature extractor with extended receptive fields capable of linking salient features from the entire sequence .
one potential problem about the vanilla Transformer, as shown in Figure 1a, is that both the encoder and decoder stack layers in sequence and only utilize the information in the top layer.