For any pair x = (x 1 , ..., x N ) and y = (y 1 , ..., y N ), P (X, Y ) ≥ P (X)P (Y ): (Equality holds when x and y are orthogonal, and Therefore, the best we can do for disjoint phrases is learn an embedding that assumes the phrases are independent.
we can map the disjoint phrases to two vectors whose computed joint probability is the product of the individual phrase probabilities.