As illustrated in Figure 2, given two input embedding matrices X a and X b , we first compute the affinity matrix A ∈ R M b ×Ma , whose elements represent the affinity scores corresponding to all word pairs between sequences t a and t b Subsequently, we compute the context-aware matrix for sequence t b as where the softmax(•) function is applied columnwise to A, and thus A b contains the attention weights (importance scores) across sequence t b (columns), which account for each word in sequence t a (rows).
matrix X b ∈ R d×Ma in (10) constitutes an attention-weighted embedding for X b .