However, when it comes to longer documents, basic attention mechanism may lead to distraction and fail to attend to the relatively salient parts.
some works focus on designing various attentions to tackle this issue (Tan et al., 2017;Gehrmann et al., 2018).