7mainly considers the discrepancy of alignment and integrity between the model output and the ground-truth, it rarely inspects grammar correctness and language fluency.
then the self-attention weights W is computed as: where U a ∈ R d×d .