For tasks with shorter sequences such as AG's News, the improvement is limited.
the improvements get more significant with the growth of sequence lengths among different datasets 4 , and the performance is particularly good in P-MNIST with very long sequences.