It shows that the number 32 (2 5 ) gives a good tradeoff between performance and computation, although there is a small gain by using more n-grams.
first, both methods rely on the word masking strategy so that the encoder can only be trained with existing word and phrase information.