(3)) penalizes more heavily reconstruction error of frequent cooccurrences, improving on PPMI-SVD's L 2 loss, which weights all reconstruction errors equally.
as it does not penalize reconstruction errors for pairs with zero counts in the co-occurrence matrix, no effort is made to scatter the vectors for these pairs.