An external short-term memory is designed to remember what has seen in the prior levels and guide the attention tuning process in current level.
in the first attention level, we concatenate R 1 0 , R 2 0 and R 1 0 −R 2 0 and apply a non-linear transformation over the concatenation to catch the general understanding of the argument pair.