Long Short Term Memory networks (LSTMs) (Hochreiter and Schmidhuber, 1997) are a variation of RNNs that are designed to avoid the vanishing/exploding gradient problem (Bengio et al., 1994).
biLSTMs take as input a sequence of words x = {x k |k ∈ N } and output a sequence of hidden vectors: H = {h k |k ∈ N } biLSTMs combine a left-to-right (forward) and a right-to-left (backward) LSTM to capture both left and right context.