Unlike in natural language corpora, the sources are independent: e.g., dog never directly appears with cat, and hence any learned relation between them could not be due to first-order information.
by defining crisp semantic categories and sentence frames, we investigate how first and second-order co-occurrence information sources are consumed and represented in terms of similarity between words by count-based and predictive DSMs.