We used several sources of text data that are publicly available and the Gigaword dataset, as described in Table 1.
we used English Wikipedia from June 2017, from which we used the meta pages archive which resulted in a text corpus with more than 9 billion words 3 .