We use the same set of hyperparameters as described in the base papers, except for the batch size with Alleviated TOI (P), where we use a prime batch size in order to prevent any repetitions in batches, as described in Section 4.
on the PTB dataset, we use a sequence length of 70 for all the models.