The matrix factorization family only uses the statistics of co-occurrence counts, disregarding of the position of word in sentence and word order.
it should be noted that the speed of our approach is acceptable compared with previous neural network language model based methods, including (Bengio et al., 2003;Mikolov et al., 2010;Mnih and Hinton, 2007), as our model uses a much simpler output layer which only has two nodes, avoiding the time consuming computation of the big softmax output layer in language model.