As the state-of-the-art Transformer model is welldesigned in the model structure, an analysis of the integration in Transformer is thus necessary.
and the context information assigned to the j th word in the sentence S i is detailed as: where α i,j is the attention weight assigned to the word and d ctx i,j is the corresponding context information distributed to the word.