Experimental results show that the additional recurrence encoder, implemented with either RNN or ARN, consistently improves translation performance, demonstrating the necessity of modeling recurrence for Transformer.
the ARN implementation outperforms its RNN counterpart, which confirms the strength of ARN.