In particular, the vector difference w i − w k is commonly used as a model for the relationship between words i and k. For a given context word j, we have The latter is an estimation of log P (i,j) P (i)P (j) − log P (k,j) P (k)P (j) = log P (j|i) P (j|k) .
the vector translation w i − w k encodes for each context word j the (log) ratio of the probability of seeing j in the context of i and in the context of k, which is in line with the original motivation underlying the GloVe model (Pennington et al., 2014).