We explored two different architectures for the student model: Simple model (MT simple): In this model, we used bag-of-word averaging to obtain embeddings for entity mentions and the patterns matching them.
we: (a) average word embeddings to obtain a single embedding for the entity mention and each of its patterns; and (b) average the resulting pattern embeddings to produce the embedding of the corresponding context.