(2018) and follow most hyperparameter suggestions as given by the in-depth study presented in Reimers and Gurevych (2017).
we use an LSTM with 256 hidden states and one layer (Hochreiter and Schmidhuber, 1997), a locked dropout value of 0.5, a word dropout of 0.05, and train using SGD with an annealing rate of 0.5 and a patience of 3.