We introduce an attention mechanism to extract such words that are important to the meaning of the clause and aggregate the representation of these informative words to construct the clause vector.
where w m is the parameter for computing attention signals and ⊕ is concatenate operation.