In practice, we repeat this process 5 − 6 times to build tokens representing longer ngrams.
we used English Wikipedia from June 2017, from which we used the meta pages archive which resulted in a text corpus with more than 9 billion words 3 .