This is the bias of unidirectional decoders and we replace them with simpler word selectors.
we only ask the model to select original neighbors for each word that loses its local word order information.