Furthermore, it will be beneficial if we process the sequence in not only a forward but also a backward way.
bidirectional LSTM (Bi-LSTM) was employed in many previous works (Chiu and Nichols, 2016;Ma and Hovy, 2016;Lample et al., 2016) to capture bidirectional information in a sequence.