As shown in Figure 1, we makeŷ ae i , the outputs from AE available to AS in the selfattention layer, as the sentiment task could benefit from knowing the predictions of opinion terms.
the self-attention matrix A ∈ R n×n is computed as follows: where the first term in Eq.