While it may be advisable to try different hyperparameters, an elaborate tuning of the RNNs was not practical given that it took about one week to train a single hyperparameter configuration for machine translation on the WMT data set on our hardware.
this is intuitively obvious because a fixed-size vocabulary cannot contain the names of all people, places, or organizations.