Here, we adopt bidirectional LSTMs (Bi-LSTMs) to take full advantages of additional backward information and enhance the memory capability.
the Bi-LSTMs contain the forward LSTMs which process the sentence s o i from w 1 to w n w and backward LSTMs which read s o i in the reversed direction: where h t denotes the representation of position t by concatenating the forward hidden state − → h t and backward hidden state ← − h t together.