In particular, the factorization's loss function is not based on L 2 , and is much less sensitive to extreme and infinite values due to a sigmoid function surrounding w • c. Furthermore, the loss is weighted, causing rare (w, c) pairs to affect the objective much less than frequent ones.
while many cells in M PMI equal log 0 = −∞, the cost incurred for reconstructing these cells as a small negative value, such as −5 instead of as −∞, is negligible.