Instead of simply averaging these vectors, we choose a linear combination of the |T u | vectors.
we apply self-attention mechanism (Lin et al., 2017) that takes H u as input and outputs a distribution α u over |T u | vectors: where W a1 ∈ R da×d is a weight matrix and w a2 is a vector of parameters with size d a .