The disagreement regularization serves as an auxiliary objective to guide the training of the related attention component.
we propose three types of disagreement regularization, which are applied to the three key components that refer to the calculation of feature vector using multi-head attention.