Intuitively, we apply KL (Kullback-Leibler divergence) as the optimized function, which describes the differences between distributions: What is more, the Equation 2 can be further reduced as following: where loss a represents the loss of attention regularization.
because a m contains fixed values, the equation is equal to we adapt loss a into classification loss loss c to regularize attention learning.