The generation of the output sequence is conditioned on the previous words and the input.
when certain sequences are very common, the language modelling conditional probability will prevail over the input conditioning.