Firstly, we observe that by introducing attention regularizations (either R s or R o ), most of our proposed methods outperform their counterparts.
aT-CaN-R s and aT-CaN-R o outperform aT-LSTM in all results; aTaE-CaN-R s and aTaE-CaN-R o also outperform aTaE-LSTM in 15 of 16 results.