For time step t in layer j, a selfattention sub-layer of hidden state s t,j is calculated by employing self-attention mechanism over hidden states in previous target layer, which are Then, a second target-tosource sub-layer of hidden state s t,j is inserted above the target self-attention sub-layer.
the queries(Q) are projected from s ′ t,j , and the keys(K) and values(V ) are projected from the source hidden states in the last layer of encoder, which are {h 1,n , h 2,n , ..., h m,n }.