The test perplexities (Table 2) and validation learning curves (Figures 2, 3, and 4) show that the competitiveness of softmax diminishes with larger vocabularies.
figure 7 shows validation perplexity versus the number of iterations for a week of training.