RNN is a prevalent family of models for many NLP tasks and has demonstrated compelling performances (Mikolov et al., 2010;Sutskever et al., 2014;Peters et al., 2018).
its sequential nature makes it heavily reliant on word order information, which exposes to the risk of encoding language-specific order information that cannot generalize across languages.