First of all, we would like to investigate how our neural network approach might be improved by incorporating information from other sources.
we think of initializing our embedding matrices with distributed representations that come from a large-scale neural language model (Mikolov et al., 2013).