The typical way of training NLMs is to maximize the training data likelihood, or equivalently, to minimize the cross-entropy objective of the following form: Training NLMs can be prohibitively slow due to the computationally expensive softmax layer.
past works have tried to use a more efficient version of the softmax such as the hierarchical softmax (Morin, 2005;Mnih and Hinton, 2007;Mnih and Hinton, 2009) or the classbased one (Mikolov et al., 2010;Mikolov et al., 2011).