With our method, the performance of a well-optimized 6block model can be further boosted by adding two additional blocks, while simply using Transformer (8B) will lead to a performance drop.
we achieve a 29.92 BLEU score on En→De translation with 1.0 BLEU improvement over the strong baselines, and achieve a 0.6 BLEU improvement for En→Fr.