Instead of training the transferred model by ourselves, in this paper, we adopt the ELMo word representations , which are derived from a bidirectional language model (BiLM) trained on one billion word benchmark corpus (Chelba et al., 2014).
this BiLM has one character convolution layer and two biLSTM layers, and correspondingly there are three internal representations for each word x t , which are denoted as {h LM t,l } 3 l=1 .