On the large corpus, the conventional NCE model doesn't even reach a perplexity under the size of the vocabulary, whereas augmenting k to 500 is not enough to reach a suitable perplexity.
increasing k reduces the computational benefit of NCE, even with k far inferior to the vocabulary size.