Then, a self-attention mechanism is applied to learn the attention weight of every word in the snippet, identifying the key words and ruling out irrelevant information.
where ω j ∈ R 1×K , p ω ∈ R n h ×1 , W h ∈ R n h ×n h , and u j ∈ R 1×n h .