The Siamese neural network with GRU was tested with a cyclical learning rate (Smith, 2015), which has the advantage of forcing the model to find another local minimum if the current minimum is not robust and makes the model generalize better to unseen data.
neither cyclical learning rate nor reducing learning rate on plateau increased the per- 3 These results are reported in Marelli et al.