The second term however can approximated using samples {u i } from the current model - 3 In (Welling et al., 2003) the term for P 0 does not appear, which is equivalent to taking the uniform distribution as the baseline model.
given a set of N samples {u i } from the model, the optimal feature to add is one that gives high scores to sampled sentences and low ones to real sentences.