Such vast corpora have led to leaps in the performance of many language-based tasks: the concept is that simple models trained on big data can outperform more complex models with fewer examples.
this new view comes with its own challenges: principally, how to effectively represent such large data sets so that model parameters can be efficiently extracted?