In standard N-Gram models the bias feature corresponds to the unigram distribution.
even the more scalable variant proposed by Mnih and Hinton (2008) is trained on a dataset consisting of only 14M words, also using a vocabulary of around 20000 words.