We then use an attention model introduced by (Ganea and Hofmann, 2017) to compress the context into a single embedding.
we define the contextual representation The weight vector α is computed using the following attention considering all k entity candidates: where A is a learned matrix that scores the relatedness between word and entity.