We found that most hyperparameters could be safely fixed the same way for all three types.
our final models were all trained in batch mode using the Adam optimizer (Kingma and Ba, 2014), with each batch covering 25 scenes given to the model in chunks of 750 tokens.