The discrepancy gets more severe as the length of the correct sequence increases.
even after the training loss converges to a small value, full inference on the training data can incur errors causing the model to be underfitted for long sequences in spite of low training cost.