Given a sequence of words p = (w 1 , w 2 , ..., w d ) with window size d, the goal of the model is to discriminate the sequence of words p (the correct phrase) from a random sequence of words p r .
the objective function of the model is to minimize the ranking loss with respect to parameters θ: , where p is the set of all possible text sequences with d words coming from the corpus U , R is the dictionary of words, p r denotes the window of words obtained by replacing the central word of p by the word r and f θ (p) is the score of p. The dataset for learning the language model can be constructed by considering all the word sequences in the corpus.