We also employ a learning rate multiplier on the gradient.
the weight update for the ith encountered training sequence during training is The learning rate function is modelled on the one used by Finkel et al.