Such vast corpora have led to leaps in the performance of many language-based tasks: the concept is that simple models trained on big data can outperform more complex models with fewer examples.
from GW and 13.20B tokens and 0.8B types from GWB50.