Given a variable-length words sequence X = (x 1 , x 2 , ..., x L ) in a paragraph, the word-level Bi-LSTM will process the input sequence by using two separate LSTMs, one process the word sequence from the left to right while the other follows the reversed direction.
at each word position t, we obtain two hidden states − → h t , ← − h t .