For flat networks with a single LSTM-layer, around 150 recurrent units yielded the best performance.
gradient normalization has a better theoretical justification and rescales the gradient whenever the norm goes over a threshold.