We have empirically found that omitting this bias term does not affect the performance of the model, while it allows us to have a more direct connection between the vector w i and the corresponding PMI scores.
let us define PMI W as follows: Clearly, when the word vectors are trained according to (2), it holds that PMI we can think of the word vector w i as a low-dimensional encoding of the vector (PMI S (i, 1), ..., PMI S (i, n)), with n the number of words in the vocabulary.