We employ a bidirectional Long Short-Term Memory Network (BiLSTM) on top of the embedding layer to capture the temporal interactions among words.
at time step t, given the input word embedding x, the update process of forward LSTM network can be formalized as follows: Where σ is the sigmoid activation function, i t , f t , o t are the input gate, forget gate and output gate, respectively.