In (Ng and Zue, 2000), an analysis of news text demonstrated that the vocabulary size would continue to grow as the dataset got larger.
it was not possible to create single large vocabulary that would eliminate the out-ofvocabulary problem.