From the results, the performance of our model is significantly improved by 0.93 BLEU points (the first two rows in Table 4) when the largescale sentence-level parallel corpus is used for the pre-training process.
when we use the mixed data of both sentence-and document-level parallel corpora 2 to first pre-train our model, the performance of our model is significantly improved by 5.26 BLEU points (the last row in Table 4).