Given a document collection (data) {V d } T d , and another collection (noise) {V n } Tn with T n = kT d , NCE distinguishes these (1+k)T d documents simply based on Bayes' Theorem, where we assumed data samples matched by our model, indicating P θ P data , and noise samples generated from an artificial distribution P n .
in this paper, we propose an efficient learning strategy for RSM named α-NCE, applying NCE as the basic estimator.