We approximate that it takes about one minute to obtain n-gram statistics from the Wikitext-103 training corpus, for example.
we set up our experiments to equalize neural network training time, and we ignore the small onetime cost of computing n-gram statistics.