The main idea of most word embedding models follows the distributional hypothesis (Harris, 1954), i.e., the embedding of each word may be inferred using its context.
x (2) ij = CO(i, j) represents context overlap, a word independent learnable scale A is adopted to relieve the potential inconformity between first order and second order co-occurrences.