ELMO-Sub provides subword-level contextual representations, which we suspect could be the cause of unstable performance.
to get the representation for a word in evaluation on word-level tasks, we follow Devlin et al.