We propose a model which explicitly prevents this by ensuring that successive context vectors are orthogonal to each other.
we subtract out any component that the current context vector has in the direction of the previous context vector.