We employ dropout on the embedding layer, LSTM layer and the penultimate layer.
although bidirectional RNN has access to both past and future context information, the range of context is limited due to the vanishing gradient problem.