Next, we gather all the information on passages by a simple concatenation of its contextual information H p and its question-aware representation H q • C: ( 2)Typically, a passage may contain hundred of tokens, making it hard to learn the long dependencies within it.
inspired by (Lin et al., 2017), we apply a self-attended layer to rearrange the information U p as: we first obtain an n × n attention matrix with U p onto itself, apply dropout, then multiply this matrix with U p to obtain an updated U p .