We performed the same configuration on our models according to the settings of the Maruf and Haffari (2018).
for the Transformer, we set the hidden size and point-wise FFN size as 512 and 2048 respectively.