The context vectors are accumulated by adding together k-dimensional index vectors that have been assigned to each context in the data -whether document, paragraph, clause, window, or neighboring words.
another refinement method is to use feature extraction, which means that "artificial" features are created from the original ones, either by using clustering methods, such as distributional clustering (Baker and McCallum, 1998), or by using factor analytic methods such as singular value decomposition.