Held-out perplexity on real data provides a quantitative evaluation of our models' performance in a real-world setting.
the goal of our models is to enable a deeper analysis of large, weakly-related corpora, which we next discuss.