We find that both using pre-initialized embedding vocabularies, and the corrupt-sampling method employed are very important for properly training our model.
the gap between results of all systems tested on both CoNLL-YAGO and WikilinksNED indicates that mentions with noisy context are indeed a challenging test.