In addition, we introduce a novel type of LM using a modified version of the standard bidirectional LSTM called contextual bidirectional LSTM (cBLSTM).
to the unidirectional model, this model is trained to predict a word depending on its full left and right contexts.