in the word encoder yields a set of attention matrices A = {A (k) } r k=1 for each sentence where A (k) ∈ R n×n .
when measuring the contributions from words to sentences (e.g., in terms of domain scores and representations) we can selectively focus on salient words within the set A = {A (k) } r k=1 : where the softmax function outputs the salience distribution over words: and obtain sentence embeddings g = Zα.