However, it has greatly improved the translation performance.
by comparing Row 3 and 4, we find that our proposed methods achieve a similar performance with the Transformer(Big) and gain a faster speed with fewer parameters.