Our pooling operation makes the conceptual simplification that all previous instances of a word are equally important.
we may find more recent mentions of a word -such as words within the same document or news cycle -to be more important for creating embeddings than mentions that belong to other documents or news cycles.