In this paper, we introduce orthogonal regularization to constrain the attention weights of multiple non-overlapping aspects, as well as sparse regularization on each single aspect.
given the sentence S with K aspects, A s = {A s 1 , ..., A s K }, for each aspect A s k , its attention weights are calculated by: where u s k is the embedding of the aspect A s k , e L ∈ R L is a vector of 1s, u s k ⊗ e L is the op-eration repeatedly concatenating u s k for L times.