Language models are trained to predict all tokens in the question, web source, and answer.
the standard Seq2Seq model only receives training signal from predicting the answer which is much less than the language model gets.