We will use the word subsampling probability g(y) above to define the cost for the pair (x, y) as the expected frequency of the pair in the training set.
we use the subsampling frequency from word2vec to define g(y) as follows: 2 Here, n(y) is the context-independent frequency of label y in the whole corpus and α is a hyperparameter.