For this reason, we ground the encoder by a language modeling component (Bengio et al., 2003;Mikolov et al., 2011).
we employ two Softmax classifiers to predict the word x i based on − → h i−1 and ← − h i+1 respectively.