After running a transformer encoder where each position can attend to every other position, a placeholder token will have a probability distribution over the output vocabulary and this distribution is informed by all other tokens in input and output.
a place-  Table 7: Average attention weights and standard deviation when predicting from left-to-right on both SHARC and DAILY DIALOG (DD) for different parts of the sequence, where α 1 is for the input sequence x, α 2 /ᾱ 2 is for the already produced sequence y and α 3 /ᾱ 3 is for the sequence of remaining placeholder tokens p. α k are the normalized attention weights across all three parts, whereasᾱ k normalizes over the second and third part.