Here we compare the two on the regular training setting where we train the model from scratch on a large dataset.
we compare the initial training performance on 100, 500, and 900 domains which are the same as we used earlier.