Moreover, we also use a least squares formulation to constrain these vectors such that w i •w j reflects the co-occurrence statistics of words i and j.
to GloVe, however, we explicitly model two factors that contribute to the residual error of this model: (i) the fact that corpus statistics for rare terms are not reliable and (ii) the fact that not all words are equally informative.