In fact, the notion of "attention" has gained popularity recently in neural network modeling, which has improved the performance of many tasks such as machine translation (Bahdanau et al., 2015;Luong et al., 2015).
let a be the magnitude of the columns in A o and a ∈ R m .