For the sentence-level Transformer, in order to make the positional encoding insensitive to the lengths of the phraselevel fragment, we set: Multi-head Attention: We modify the selfattention mechanism in the encoders and decoders by setting a different receptive field for each granularity, as illustrated in Figure 2.
for the phrase-level model, we restrict each position in the encoder and decoder to attend only the adjacent n words (n = 3), so as to mainly capture the local composition.