Given the context of mention m, we form its representation from involved contextualized word vectors with a mention-aware attention mechanism where C is the number of contextual words, and a c i is defined as a c i = Softmax(e c i ), where where ⊕ represents concatenation, and v c ∈ R da and W c ∈ R da×(2dr+1) are trainable parameters.
our Model* is a variant where we replace ELMo embeddings with Bert (large cased model) contextualized word representations.