We deviate from the standard decoder by removing the selfattention mask that prevents left-to-right decoders from attending on future tokens.
our decoder is bi-directional, in the sense that it can use both left and right contexts to predict each token.