The training criterion is that n-grams that are present in the training corpus like x must have a score at least some margin higher than corrupted n-grams likẽ x.
: We minimize this loss stochastically over the n-grams in the corpus, doing gradient descent simultaneously over the neural network parameters and the embedding lookup table.