The coefficients c ij is computed using a softmax function, and M j=1 c ij = 1.
the major difference between DR-AGG and self-attention (Lin et al., 2017;Yang et al., 2016) is that the query vector of self-attention is task dependent trainable parameters learned during the training phase, while the query vector of DR-AGG is each input or output capsule which is instance dependent and dynamically updated.